Sounds to me like you're either:
A) Targeting your own people
B) Your party members are getting charmed and thus, attacking their own
C) They killed someone and didn't autotarget a new enemy
D) Any combination of the above.
Man, I've been playing on hard, and while I get my ass kicked if I forget to use any semblance of strategy at all, I've been having some fairly smooth sailing, with only one single optional fight just crushing me 5+ times... though I just beat that a couple hours ago.
This combat system plays, to me, exactly like BG series, because I took the AI off all the time.