That's just it: swords are not heavy. A german Zweihänder for example - a real two handed sword, as long as its user - typically weighs at most 3 kg (about 6.5 lb), a Scottish Claymore about 2.5 kg (5.5 lb). Swords over 4 kg do exist, but they're rare and largely ceremonial - and generally considered unwieldy.
Aside from weight, swords are very well balanced. They are not sharpened baseball bats.
As for war hammers, does this look like it weighs a ton: http://www.medievalwarfare.info/pics/warhammer.jpg ?
That's correct. Bows each have their own draw weight, and if you can't meet that, you can't use the bow efficiently or at all. Historical longbows sometimes had pretty astronomical draw weights, that's why the best archers were trained from a young age.