So you're rejecting them in the first meeting then finding and killing them later? This would be useful to know because a few of them have basically build defining weapons.
I had been inviting them, though they never enter the party because it's solo, then like Boeroer pointed out I found I was unable to do enough damage to kill them once the were considered an ally.