In creative mode guards items from their inventory will despawn when opening their inventory for the first time, but this does not happen when in survival mode.

Dont be in creative lol

So if I want to use this and setup a village for players or a town outpost... I need to leave creative every single time just to change their equipment? Then change my gamemode back to continue working?

Yes, because i have no idea whats causing it