It is my understanding from a Kotaku(?) article that when you lose a hat in a fight, or get hit, or it flies off that it is not in fact lost. I thought I read that there is an easy to overlook menu, buried maybe two layers deep in the wheel, where all the hats you've ever acquired are permanently stored. I could be wrong.
EDIT: Here is a different link that says similar.