I had a similar issue with inventories overwriting others, might be part of the same origin, as in your case the first inventory is being cleared before transferring back to the other world.
Details for my case:
We're trying to reduce player health to half a heart utilizing AttributeModifiers on items, in our case golden boots.
They're unbreakable and have the enchants curse of vanishing/binding. The modifier is limited to when they're worn on the feet. All of these attributes are hidden.

Nothing goes wrong when the user has these in any slot where they have no effect however if they're worn as intended and the player moves to a world which does not share the inventory, first the inventory gets cleared and then the inventory from the world with the boots is transferred over, this includes the ender chest.
It seems to be somewhat flashy, you're actually able to see for a split second, that your inventory is empty before you then have the items which you should not have.

Originally posted by @Skellett1 in #382 (comment)

I also just found this in the console, it appears that MV Inventories restores the health before changing the inventory, this leads to the error that the game expects a limit of 1 health, yet mv inventories tries to set it to something which exceeds this limit. I think this can be circumvented by checking for any player stat changes within the new inventory before swapping over. currently this results in the prior inventory to be restored and all of the original items including the ender chest to be lost:

[23:37:05 ERROR]: Could not pass event PlayerChangedWorldEvent to Multiverse-Inventories v4.0.0-SNAPSHOT-bUNKNOWN
java.lang.IllegalArgumentException: Health must be between 0 and 1.0, but was 20.0. (attribute base value: 20.0, player: Skellett1)
at org.bukkit.craftbukkit.v1_16_R1.entity.CraftLivingEntity.setHealth(CraftLivingEntity.java:115) ~[patched_1.16.1.jar:git-Paper-49]
at com.onarandombox.multiverseinventories.share.Sharables$5.updatePlayer(Sharables.java:183) ~[?:?]
at com.onarandombox.multiverseinventories.ShareHandler.updatePlayer(ShareHandler.java:139) ~[?:?]
at com.onarandombox.multiverseinventories.ShareHandler.completeSharing(ShareHandler.java:88) ~[?:?]
at com.onarandombox.multiverseinventories.ShareHandler.handleSharing(ShareHandler.java:59) ~[?:?]
at com.onarandombox.multiverseinventories.InventoriesListener.playerChangedWorld(InventoriesListener.java:209) ~[?:?]
at com.destroystokyo.paper.event.executor.asm.generated.GeneratedEventExecutor107.execute(Unknown Source) ~[?:?]
at org.bukkit.plugin.EventExecutor.lambda$create$1(EventExecutor.java:69) ~[patched_1.16.1.jar:git-Paper-49]
at co.aikar.timings.TimedEventExecutor.execute(TimedEventExecutor.java:80) ~[patched_1.16.1.jar:git-Paper-49]
at org.bukkit.plugin.RegisteredListener.callEvent(RegisteredListener.java:70) ~[patched_1.16.1.jar:git-Paper-49]
at org.bukkit.plugin.SimplePluginManager.callEvent(SimplePluginManager.java:607) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.PlayerList.moveToWorld(PlayerList.java:869) ~[patched_1.16.1.jar:git-Paper-49]
at org.bukkit.craftbukkit.v1_16_R1.entity.CraftPlayer.teleport(CraftPlayer.java:813) ~[patched_1.16.1.jar:git-Paper-49]
at org.bukkit.entity.Entity.lambda$teleportAsync$0(Entity.java:178) ~[patched_1.16.1.jar:git-Paper-49]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_252]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_252]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_252]
at org.bukkit.entity.Entity.teleportAsync(Entity.java:178) ~[patched_1.16.1.jar:git-Paper-49]
at org.bukkit.craftbukkit.v1_16_R1.entity.CraftPlayer.teleportAsync(CraftPlayer.java:751) ~[patched_1.16.1.jar:git-Paper-49]
at com.earth2me.essentials.paperlib.features.asyncteleport.AsyncTeleportPaper_13.teleportAsync(AsyncTeleportPaper_13.java:13) ~[?:?]
at com.earth2me.essentials.paperlib.environments.Environment.teleport(Environment.java:92) ~[?:?]
at com.earth2me.essentials.paperlib.PaperLib.teleportAsync(PaperLib.java:87) ~[?:?]
at com.earth2me.essentials.AsyncTeleport.lambda$nowAsync$3(AsyncTeleport.java:211) ~[?:?]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_252]
at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_252]
at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_252]
at com.earth2me.essentials.AsyncTeleport.nowAsync(AsyncTeleport.java:182) ~[?:?]
at com.earth2me.essentials.AsyncTeleport.teleport(AsyncTeleport.java:278) ~[?:?]
at com.earth2me.essentials.AsyncTeleport.warp(AsyncTeleport.java:416) ~[?:?]
at com.earth2me.essentials.commands.Commandwarp.warpUser(Commandwarp.java:100) ~[?:?]
at com.earth2me.essentials.commands.Commandwarp.run(Commandwarp.java:47) ~[?:?]
at com.earth2me.essentials.commands.EssentialsCommand.run(EssentialsCommand.java:168) ~[?:?]
at com.earth2me.essentials.Essentials.onCommandEssentials(Essentials.java:615) ~[?:?]
at com.earth2me.essentials.Essentials.onCommand(Essentials.java:525) ~[?:?]
at org.bukkit.command.PluginCommand.execute(PluginCommand.java:45) ~[patched_1.16.1.jar:git-Paper-49]
at org.bukkit.command.SimpleCommandMap.dispatch(SimpleCommandMap.java:159) ~[patched_1.16.1.jar:git-Paper-49]
at org.bukkit.craftbukkit.v1_16_R1.CraftServer.dispatchCommand(CraftServer.java:794) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.PlayerConnection.handleCommand(PlayerConnection.java:1908) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.PlayerConnection.a(PlayerConnection.java:1719) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.PacketPlayInChat.a(PacketPlayInChat.java:47) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.PacketPlayInChat.a(PacketPlayInChat.java:5) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.PlayerConnectionUtils.lambda$ensureMainThread$0(PlayerConnectionUtils.java:23) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.TickTask.run(SourceFile:18) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.IAsyncTaskHandler.executeTask(IAsyncTaskHandler.java:136) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.IAsyncTaskHandlerReentrant.executeTask(SourceFile:23) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.IAsyncTaskHandler.executeNext(IAsyncTaskHandler.java:109) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.MinecraftServer.aZ(MinecraftServer.java:1136) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.MinecraftServer.executeNext(MinecraftServer.java:1129) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.IAsyncTaskHandler.awaitTasks(IAsyncTaskHandler.java:119) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.MinecraftServer.sleepForTick(MinecraftServer.java:1090) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.MinecraftServer.v(MinecraftServer.java:1004) ~[patched_1.16.1.jar:git-Paper-49]
at net.minecraft.server.v1_16_R1.MinecraftServer.lambda$a$0(MinecraftServer.java:177) ~[patched_1.16.1.jar:git-Paper-49]

same error occurs when changing game mode whilst wearing the boots