I'm also a single playthrough kind of guy and what bothers me is the risk of missing some really cool content because I didn't have a certain companion in my party during a particular encounter. Not just a throwaway line but information that gives me deeper understanding of characters or conflicts, or just a different solution to the encounter.
On the other hand I'm not sure I would actually enjoy the game more if I had all the companions present in the background at all times. To have to leave a few companions behind kind of enhances my role as leader.