So I have finally found the solution to this problem while actually looking how to fix the exact same issue for Tyranny which I recently purchased (since Deadfire was unplayable)
Anyway here it goes a quote from steam community:
What is the problem you ask? The graphics card was seated in a PCI-E slot on my motherboard that would only run at full power (x16) under the right circumstances. In this case it was reserved for SLI-mode (dual-GPU) configurations, and was running at x2 mode. What this means in laymans terms is that the data lanes between the GPU and my system were at 2 instead of 16, and it was choking like a movie stream over 64k internet. What is amazing about this specific problem is that it'll only affect games that send a lot of visually heavy data to the GPU, like Tyranny. Games like Battlefield send less texture data than Tyranny, despite being so much more graphically intense, and so I could play BF1 at 60FPS no worries, but not Tyranny. That still points to really bad optimisation on Obsidians part, but it is only a problem because of my issue.
As it says someone had a problem that the GPU was plugged in to the wrong PCI-E slot which in my case was a 2nd Slot in an ASUS X99 Pro motherboard which does't function properly. Only slots 1,3 & 4 allow full potential for GPU. So now that it is plugged in to slot 3 the game runs smooth and well and is playable.
N7Cross's post in Stutter/FPS Drops was marked as the answer