For instance, if the first spell I cast in combat has Average cast time, will there be a big difference between having 18 dex and 4 dex?  Or does having more dex allow you to cast more spells during combat, but their cast time is static?

 

 

Sure does, and by quite a noticeable difference, even between 10-20 Dex. Cast the spell faster, recover faster. Dex is different from attack speed % and is far superior.