I think a good approach should not affect DPS or speed at all, but instead limit the flexibility of the class. So, my idea would be to limit spellcasters in their spell choices depending on the type of armor.
Spells are divided into "difficulty"-tiers. This is not to be confused with the spell levels.
Each spell gets a certain difficulty tier applied to them:
1) can be cast with all types of armor
2) can be cast with chainmail and below
3) can be cast with leather and below
4) can be cast with light armor and below
5) can only be cast in cloth armor
I see where you're going with this. It might be useful to give some examples of existing spells that would go with each armor level. The real make or break for your idea is whether the spell system as is (or tuned in a new way) will work with difficulty tiers instead of time penalties, e.g., are there enough, say, level three spells to cast in plate; is there enough of a difference between plate and chain to make chain a viable choice etc.
How about non-caster classes? I assume physical attacks still have a speed penalty? How about other class abilities?