I never understood why this bothered people, I mean, it's annoying I guess but do you really notice it when you're playing?
A long time ago, in my youth, my parents fridge broke down. Slowly. It started by making this weird, loud whirring sound in the middle of the night. After a while, it just stopped working.
A friend of mine bought an Xbox 360 recently. He invited me over to play Call of Duty: World at War in co-op a couple of weekends ago. When I got there and we started playing, I heard that loud, whirring noise again so I asked him if his fridge was broken. He said, "No, why?". It was his Xbox 360.
I honestly thought it was broken when I heard that sound! So YES, I was definitely bothered by it. And surprised! I had heard the horror tales about the sound level of the Xbox 360 on the net before, but I thought it was mostly fanboi war exaggerations..
Well, yeah I get that it's loud or whatever, but people say it's reason to not buy the console, like, maybe in your case it makes more sense because you have loud-fridgephobia, but for the average person I don't see it as such a horrible thing