I think there should be a family with the name "Legen", who runs a Dairy, and you should find a cache of weapons buried on their farm somewhere, if you solve the mystery of their location (it will be a rather elaborate mystery).
However, in order to be worthwhile, they have to be significantly better than everything else. There's no point getting excited over a +4 longsword when you can buy a +3 longsword in the shops (and a +5 longsword in the next location you visit).
True true. I will add that the "we'll just make it +17" approach shouldn't be used to make them worthwhile, since that just ends up still being bland, but OP. It's better when they do something really interesting/unique that ends up making them worthwhile, rather than just having exorbitant amounts of pure attack-boosting qualities (accuracy/damage upping). I'd honestly prefer to find legendary weapons that are "plus nothing", because they're so unique they can't even really be compared to other, "mundane" weapons on the same scale.
It should be kind of like apples and oranges. Because, if it isn't, there's inevitably something else that's almost the same thing. "Oh, I could enchant this other sword with fire, but this legendary sword of the Fire Paladin, Rouark, does fire damage, but more
damage than what you can enchant a sword to do."
When the only different is quantity, that doesn't make it very legendary. It just makes it... weirdly multiplied? That said, it could still easily be a +something weapon WITH some awesome, unique attribute. Maybe the person who used it was wielding a well-crafted weapon (+something) when it became imbued with whatever makes it unique. That'd be fine. I don't think it has to have absolutely nothing in common with regular weapons. It just needs to be more than a regular weapon with shifted numbers.