I have all the symptoms that other people described, like delayed voice over or lag when opening the inventory. Here is what I tried so far:
Starting game as Administrator
Enable / Disable vsync
Run game in window / full screen
Increase / decrease process priority
Change Core affinity to use 1,2,3 or 4 cores
Delete all portraits in \PillarsOfEternity_Data\data\art\gui\portraits\player
I am running Windows 7 and a ATI card, with the drivers beeing a few months old.