TES IV didn't break any lore, aside from not making Cyrodiil a jungle but generic fantasy land and messing up the names of Daedric realms (which was a case of writers passing the wrong notes to programmers, apparently).
In fact TES IV introduced some interesting new ideas regarding the Towers & their focusing stones (White Gold Tower + Amulet of Kings, Red mountain + Heart of Lorkhan, etc). Kirkbride wrote Nu-mantia Intercept after Oblivion.
If TES V deals with finding a new Emperor before the EMpire crumbles, for instance by tracing old Septim bloodlines (surely there were tons of bastard children created throughout the years) all over the Empire it could be pretty interesting.