When there are armor pieces in the Totem of Holding and the player is already wearing pieces of armor, the armor from the totem will replace the one already worn.

https://pastebin.com/Vwk71Xgk

This is actually intended, I misread the issue. The armor you had equipped when hitting the totem is dropped or added to your inventory.