I generally agree with that, but with combat the primary focus of many RPGs you can easily see why this is the case. I think the idea of "true" wilderness (areas yet to be mapped or explored by NPCs) can be very interesting, in opposition to the "perpetual state of semi-wilderness in every town's backyard" that many RPGs fall under. If you have frontier regions, that makes for stimulating wilderness, but transforming the little forest between two peaceful villages into a monster habitat reserve is a bit dubious.
Speaking historically, if you're an unarmed peasant (as if there's another sort) being alone outside of town (or hell, alone in town after dark) was very dangerous, between highwaymen and the chance of running into a bear or a wild boar. But there's little in a peacetime environment which should challenge a lawful armed party. After all, any threat big enough to attempt to take on a half-dozen armed, well-trained adventurers would pretty quickly run afoul with the local lord, and have some expedition sent out to wipe it clean. Only in areas which are nearly impassible (swamps, mountains), or in flux (like the Scottish Borders in the real world) was there not a quasi-monopoly on larger-scale violence by the state.
So how do you fit this into a fantasy setting? I'd say the following.
1. Isolated, solitary monsters are fine. Presume something like an ogre keeps a low profile and usually makes sure to pick off only a few weak travelers on occasion.
2. On the other hand, something like a large war band of Orcs or Goblins is just silly. Similar bands of humans (who can at least blend in) didn't survive close to settlements except in severe times of political flux. How a group of hostile demihumans would is beyond me.
3. However, an exception can be made for groups of "monsters" who keep to a confined area, and aren't worth rooting out. Say a dungeon full of hostiles. Or a haunted city. Or a high mountain valley with wild tribes within. There are any number of scenarios, but for the most part, these groups have to be passive - if they're actively threatening local cities, they would have been pacified already.