I've got the same problem with changing the party at Caed Nua
Running Steam version 3.03.1047 without the White March expansions.
Party member thats causing issues for me is Aloth - can swap out other party members fine, but if I load a save without Aloth and try to add him it glitches, or if I load one with him in party I can only remove party members. Played about 35 hours on v3.02 before the update with no issues.
Verifying cache and Re-installing the game hasn't helped for me.