So, just to tell you my party is really ok now thanks to your advice.
I got my barb as main, doing lot of aoe dmg, and usually not having issues
Eder, as main tank, pretty straightforward
Aloth built on magic weapons spec (blast etc), so that he can be useful without spells
Durance with Scion of fire, he is the only one a bit squishy, was wondering if I should give him some heavy armor instead of his robe
Sagani, with a heavy pet build, very fun to see this fox deal a ton of dmg when going out of stealth
I intend to stay at 5 ppl to level faster.
Thanks again all !