I have the same problem on PS4. Loading times are sometimes nearing 1:50 and sometimes, though not often, the game freezes during the loading process.
I had simillar problems before with Torment: Tides of Numenera, a game based on PoE technology, but not to that extent, since the game is much shorter, i beleive.
It's strange that this flaw has never been addressed in any patch. And not that it is the only annoying problem the game still has.
I beleive that I'll finish the game anyhow, but with Deadfire coming out for PS4 in a couple of months, I'm not really sure that I'm ready to subject myself to another game as technically flawed as PoE I.