Pulled up a Level 6 character that was very early in Act 1. They had just started exploring Neketaka and hardly done any quests.
Got The Honored Guest invite immediately on map change to the World Map.
So it is in fact possible to get the invite VERY early. How early is a matter to be seen.
Thinking about this now, and I don't have time to test it as I am literally running out the door after I hit "Post", I think it might trigger when one enters the World Map for the first time on a save where the trigger (whatever it is) has already been set.
My first test was with a save where the party was ALREADY on the World Map. I got the invite when I left the Deck of Many Things and hit the World Map again. My second test was when I left Queen's Bearth and entered the World Map. Got it immediately again.
So for folks who are already past Port Maje I would see if going to the World Map gets the invite.