After starting a new party (in this case Quest), you can no longer reduce / remove party members, nor replace them with someone else. This blocks you from using & leveling varied members over time, or adapting to encounters.
Example: You start with 4 members. In the initial screen, you can add / remove correctly.
Finish first adventure. Try to change party members - You can no longer remove nor replace anyone.
General (For all issues)
What device type are you on? Android
What version of the OS are you running? Android 6.0.1
What model is the device? Galaxy S7 Edge
Gameplay
Is pass & play on? No. Is permadeath on? No.
Tutorial, Story Mode, or Quest Mode? Quest
Characters in Party (Lini, Seelah, Harsk, Seoni)
Rest is not applicable (it's not during an adventure, but after).