Since the latest patch and/or the White March coming out, whenever I recruit Hiravias (may occur when recruiting other party members as well, I'm uncertain) I can no longer change my party. I can remove members, but when I try to add other party members, it doesn't work.
It can be fixed by disabling the auto-levelling of newly recruited party members, so It has something to do with that.