It's always hard to do a "they were men of their time" and have that aspect of it's really awkward to judge people by the morality and ethics of a time several hundred years different.
As I posted in the weird but interesting thread, Columbus was judged horrible by the people of his own time.
However, with the founding fathers (and speaking as a Brit), they had equal amounts of genius, flair, pettiness, hypocritical attitudes, and innovative beliefs. But the Americans have really pushed the mythologising sainthood aspects of them over time. So it's kind of interesting to see some of this peeling back of who they actually were.
But still, it's all trying to examine people in the context of their time and the cultures around them, not the standards we have now.