That depends. Some people I know, for instance, are pretty conservative when it comes to music (I'm using conservative in the sense that they frown on anything that doesn't sound like, say, Norah Jones, or repetitive jazzy records with lifeless, predictable tunes), and many things in my music library are considered "weird" by them. By comparisons I consider some of things in my library to have once been weird and exciting to me, and nowdays aren't that much (an example being Korn, which I basically stopped caring for around the forth album).
I'm sure there's a lot of weird music out there which I haven't heard, and that I'd probably like to. My tastes are far from being that diversified, in fact, and I resent that. I'm always on the lookout, though.
As for "off-the-radio ****", I don't recall there being many radio stations around here that play Jon Spencer, Portishead, Nine Inch Nails, Deftones or even Housemartins (weird considering Housemartins were a pretty good pop band), just to name a few. Even if something is "off-the-radio", though, that doesn't lessen neither it's value or my appreciation of it (examples being REM or James).