Damn,
Maybe it might help, I managed to get my save back.
I bought the premium upgrade on battlenet (yeah I know, had my reasons), that worked with my gamepass subscription, to play the early access, the game was installed through battlenet. Managed to play quite a bit with saving and loading without issue until the problem appeared.
When you launch the game through battlenet, you can chose either your bnet account or Xbox account, on the game main menu. I guess it might get be the same for steam? When the problem appeared, neither profile had my saves. I tried a new game using my Xbox profile and noticed that the save button was greyed out, while using my bet profile, it would work. Weird.. came to the logical conclusion that it was some sort of authentication issue on Xbox side.
I then tried to install the game on gamepass directly instead, and launch the game from there. For some reason the game crashed on launched several time with an error code (authentication issue). After fidling around and a some google searches, managed to launch the game. And, magic, my saves were all there.
I guess that everything points out to an issue on Xbox side...