Deep Blue did not make its moves immediately. In the first game it was a bit more mechanistic, not using more than 3 minutes for a move, but as the game came to a close, there were moves that were taking as many as 6 minutes. Some moves took as much as 15 minutes.
Kasparov's issue was that Deep Blue wasn't designed to be a good chess player, but rather it was designed specifically to be Kasparov.
He also regretted playing the final game, as he wasn't really mentally into it (as evidence by his conceding defeat after only 19 moves). The big advantage Deep Blue had was that it didn't get fatigued. Kasparov spent more time analyzing the games than any other game he had taken part in, and by the end he had lost his fighting spirit.
He also admits that his prior experience with computer chess players led him to some false assumptions about Deep Blue. This was part of the reason why he lost Game 2 (it also led to him speculating that Deep Blue was being tampered with, since Game 1 was more in line with how computer chess players tend to play). The fact that it was making moves that he absolutely did not expect shocked him, and added to the pressure. That's what psyched him out.
<{POST_SNAPBACK}>
I suspect your expansion on my point was concerning my poetic licence over "immediate" ... I don't know about you, but I've played many computer chess opponents that take hours, days ... even PBEM games.