If they're slowed down, then by extension I'm sped up
If they're moving at 1/2 my speed, then I'm effectively sped up to x2 their speed
so while it might not make me swing faster, it'll make them swing a heck of a lot slower... and so it's the same effect