I just removed the mode once and re-applied it, but my collection was initialized.

If you removed the mod and loaded your game afterwards, the content will probably be wiped, but the same goes for any other modded content. Identity data is saved to entity NBT, so it's not going to save if you don't have the mod installed.