Games Performance After 1.0.3 :: Grounded Grounded Discussions (steamcommunity.com)
This thread on Steam covers both 1.0.3 and also 1.0.4, seems it's been happening for a lot of people. I saw a ton of rotten food around the entrance to the black ant hill as well. As a band-aid you can just pick up all the food and then "trash" it and block off the path with walls, it's probably only a temporary fix though - I fear it's only going to pile up somewhere else.
I trust the good folks at Obsidian will sort it out soon enough.