As designed, when switching between the two weapon sets using a key/button, there is an animation delay and there even exist a level 10 ability to reduce that delay (without using the Inventory window in combat).
I often use the Inventory window in combat and noticed I can change weapons in an instant. The only delay before attacking possible is if a firearm was not loaded (which is perfect, as designed).
Although super efficient for me, this of course does not seem like the intended design (considering the level 10 Ranger ability). That said, I could be wrong and it could be an illusion. If there is a slight delay/animation after changing weapons via Inventory, I really do not see it.