I am experiencing a very similar issue. Primary player is not AFK or dead though. Anytime I finish a wall, floor, or other building piece it becomes invisible. I can finish a floor piece, walk onto it, fall through and appear to be on the ground below while the primary player sees me standing on the finished floor. As soon as I move or jump onto a non-bugged finished floor piece, I suddenly appear in the correct place.