They might not've known that they'd fallen (or risen), as the Vrook stated - people followed you beyond reason and sense..
although there should've been some alteration to their behaviour I think it would've been too drastic for them suddenly behave in the exact opposite. As I think it was too drastic to be able to pull them all the way up, or down, their allignment bar. People don't change that sudden, at least not that fundamentally.