This was happening to me as well. Occasionally things would work on the first click, usually it would take 2-3, and sometimes it would require a dozen or more.
I managed to fix this by turning off my wireless keyboard, which was damaged by spilled whiskey and subsequently is always sending input (essentially, it functions as if a certain useless key is always being held down -- I forget which one). It's usually not a problem, but for some reason was interfering with mouse input in this case.
There are keyboard testers available online. If your problem sounds like mine, I'd recommend using one to make sure your keyboard isn't sending any input that you're unaware of.
