I didn't mind the control scheme actually.
After a while I was able to perform the actions while still watching the actual action on the screen, which made it seem a bit more cinematic.
Also, the simon says reaction, particularly the last one or two sequences, typically made sense for what your character was doing. For example, when it had you pushing down, your character would often end up ducking in the scene just afterwards.
I really enjoyed the game, and have actually played through it 3 separate times now.