If you have it on casual or whatever the easiest setting is, you can't hurt your structures when hitting it. At least that seems to be the thing now. I definitely remember destroying my buildings while fighting something around it, usually ants. Xbox one.