...tempers?
I think the charisma thing has more to do with the fact that dwarves are naturally reclusive, not exceptionally social and differ quite a bit from the normal humanoid beauty archetype.
I mean, sure you can offset ugliness, short stature and xenophobia somewhat with a good personality, but that still doesn't give you a charisma that compares to say a half-elven bard, succubus or a trained agent.