I am having this issue as well.
I even went so far as to create a junction from my HDD to my SSD and deleted ALL of my savefiles but one.
Loads are upwards of 20 seconds every loading screen and I'm about 20 hours in.
My C:\Users\***\AppData\Local\Temp\Obsidian Entertainment folder is about 15 MB
My C:\Users\***\AppData\LocalLow\Obsidian Entertainment is now 45 MB
Going to try to create junctions for these as well and see if it solves the issue.
Hardware for reference
Win 7
Core i7 940 overclocked to 3.51 GHz
6 GB Ram
AMD 7970 3 GB
1920x1080 resolution
HDD for game and saves is a Samsung EVO 850 500GB with 80% free drive space.
Game is installed and all save and temporary files have had junction points created to point to the SSD.