Known Issue, and I also stumbled across this. It's actually not even the Windows language setting, but the locale that influences the initial language setting of the game.
You can change it to English in the game's options fortunately
I hope they change the language setting logic until release (however, since I'll be using the DRM free version, for me at least the Steam language setting won't be relevant then).
samm's post in (v278) Forced German language settings was marked as the answer