Topic. I've cleared the first couple of floors with my level 5ish party but I'm certain that it will increase in difficulty as I progress down.

 

Can anyone tell me the lowest level they were able to complete the Endless Path on Normal? The quest journal makes it seem like a normal sidequest that can be tackled immediately after acquiring the Stronghold (which I claimed before completing Readric's Hold).

If you're playing on hard, you can probably get to about the eighth level at your stage of the game -- with a bit of a challenge, mind you. Less so on lower difficulties. After that, things will be a bit overbearing. Once you hit level seven or eight, the rest of the dungeon should be fairly manageable.