Something that really bothered me in D&D (especially DDO; terrible scaling there) was how frequently spells are resisted. So I'm looking at the accuracy of the spellcasters. Of course, what's not as easy to compare are the accuracy bonuses in each individual spell. A "very low" base accuracy could well be made up for there.
Is it much of a concern in PoE?
I'm not sure I'd call the Chanter a caster, anyway. From the sounds of things using your invocations is a rare occurence, and the best ones don't target your enemies.