Um, an in-game solution, please. I didn't particularly care at first, but that's because I didn't know it'd continue happening. When I came back after side-questing for awhile, they began attacking me (first time since it'd first happened), and have ever since.
Is there anything that can be done in-game to change this?