I've only noticed this with my cipher, mages and clerics seem to be OK. Quite a few times my Cipher will go through the whole motions of casting a spell, voice, hand waving, spell glow, only to then go back to reloading his crossbow. No spell is cast, and no focus is lost.
Is this a bug or something that can just happen with this particular class?