Exactly. It's all an abstraction used as an aid in understanding the outside world. Why should it trouble us if some of the constructs are more abstract than others?
Another example: A googolplex (i.e., 10^(10^100) fits right in with the "real" counting numbers. But it is also a higher number than there likely are atoms in the observable the universe. Doesn't that make it imaginary, too?
This is an example of one of my "Getting Lost in a Wiki Wonder Land" type situations where I was really high, then wikied googol, googolplex, etc. Blew my mind to think that it (googolplex) represents a number higher than the amount of atoms in the universe. I wondered how that can be possible, then, seeing as that was too mindboggling, wikied all the numbers used in modern western mathematics (0-9) and read about how they all operate and whatnot, which led me to theorize that how each number behaves individually somehow was connected to the innerworkings of the fabric of space time. Then I ate some nachos and went to sleep.