Same/similar issue happend to me, v3.03.1047 Steam, Linux.
I can't remove characters from group/party. When I am in Caed Nua and I remove or switch characters in the party, it does change in the group/party screen, but not in game itself, neither in characters avatars in game. It is always still 6 members party.
I've noticed it first (that something bad is happending) when I tried to invite Hiravias to the group. To add him to group I decided to remove Eder, Eder has been removed but Hiravias teleported/moved from his original location to the middle of Stormwall Gorge. I was left with 5 members party, officialy, because there was no Hiravias icon in the group, but when I moved to his character and talked to him, he acted like member of the party already, except I couldn't control him too.
After issue with Hiravias I decided to load earlier game, I've cleared the area and I saved the game before talking to him. It is currently my earliest save. Could invite him second time, decided to remove 1 member from party in Caed Nua to invite Hiravias. That's how I found out that I can't remove party members.
The last party member I've removed/switched was when I meet Pallegina, I've removed Kana and invite Pallegina. Unfortunately I don't have such old savegame.
Sending what I got, 6 saves, but actually only the one where I stand in front of Hiravias is important: https://dl.dropboxusercontent.com/u/88194181/PillarsOfEternity.zip