[1.16.5] Memory Leak with backpacks
Speiger opened this issue ยท 4 comments
This is more of a bug that "can be fixed" by the user but is honestly not a job of a server admin to deal with.
I created a server running with this mod.
And found out that this backpack mod uses WorldSavedData to store the backpacks contents.
Yes that is a way to solve the problem of backpacks storing to much and causing a instant ban.
But you basically lose control over "memory" since you can't validate the loss of backpacks anymore and have to keep all backpacks in ram if you want to or not.
Now that isn't a problem, whats the problem is that after pregenerating a world and also having played a bit already we have 1600 backpacks in our Ram that shouldn't exist?
Why? Musical Mobs!
Our WorldSavedData at the moment is 50-60KB big. now for a save file that is not a problem, but sadly these are highly compressed and will already eat a couple megabytes without any use they are just garbage.
So this is a really big problem but also not a easy one because every mob that spawns with a backpack basically creates more garbage.
And mobfarms are not popular at all...
So I came up with a fairly simple solution.
I assume you guys have a way of "copying" backpack. If not implement one.
The solution is have a predetermined amount of backpacks (basically generate all possible backpacks that mobs can spawn with) or have a cache that tests if a backpack was already generated by the Mobs and just recycle that.
And when a Mob Drops its Backpack it basically just copies into a new one.
For music disc mobs this would result in: 1 backpack per music disc and no matter how many million mobs you have you wont have duplicates unless the mob actually drops one (on drop or on opening i think opening is better they basically create a copy that you wont have shared backpacks)
I hope that helps fixing this memory leak.
Well, this mod has been running on many servers for a very long time and this is the first time I am hearing from anyone about memory leak so something is either the math is wrong here or this is a new issue.
And actually I took a look at the cleanup code and in 1.18 the event that is supposed to be triggered doesn't get triggered which looks like forge issue. You have never mentioned which version you are on so I am going to guess that you are on 1.18 and you're hitting this issue.
Anyway I feel like you just want to rant here and shame someone else's code so I am going to close this and fix the event related issue once forge has the fix and I have retested that it works.
There is actually quite a number of different combinations of these backpacks (just quick calculation gets me 400) and jukebox backpacks need to be separate backpacks anyway otherwise there would need to be a lot of custom code that would completely break the implementation as it is currently.
But there's actually code in place that is supposed to clean up all the backpacks that were added to mobs when these mobs are killed (unless the backpack actually drops) and when they just despawn.
So this seems like a case of the code either having bug or some mod not triggering these events when the mod removes mobs from existence.
I will take a look at what's happening there and perhaps see if there's a way to remove these backpacks even in cases where some mods may misbehave.
Anyway in the mean time you can pretty easily use /sbp removeNonPlayer
to remove all backpacks that were never accessed by players.
And just for reality check I have checked memory allocation before and after clean up of these mob backpacks in the world that I play - clean up removed 30KB from 60KB file on disc and CompoundTag allocation (which is the only thing this really is) in the memory went down by 170KB so yeah in your case it would be at most 340 KB of additional allocation in memory if all of those backpacks were only from mobs and no player hasn't used a backpack yet. So I will be taking one more look at this, but I don't see how this could be a huge issue.
So just to math out the ram.
First HashMap. The hashmap is 25% larger then the stored entries. Which is a pointer array so 8 bytes * size, and rounded up to the next power of two.
Each HashMapEntry is 1 int (4 bytes), 3 references (24 bytes).
So without any data from you and ignoring java class overhead so just raw data.
1661 * 28 bytes (entries) + 4096 array (because 2048 is to small already)
Will result into 46.508 + 32.765 => 79276bytes (80kb)
And now we come to the horribly unoptimized NBT compound storage of backpacks.
First each key is between 30-80 bytes (the first parameter you use as identifier)
Which a musical backpack has 16 from.
Then ofc each NBT compound is a wrapper. That means you get the class overhead of each variable. Though MC recycles that but its still 8 bytes for each number within -127 to 128
then long strings are way worse for data.
Which your modid sadly is.
That means you are way within to 1-2KB within each backpack.
And that doesn't account the hashmap arraylist "minimum size" allocation.
So my guess that your 1661 backpacks are between 10-50MB of ram usage. Maybe even more if you account for java Class overhead.
Now have a mobfarm and you get into the 10k radius and you can multiply that ram usage by 10 and you are by 100MB half a GIG and it will NEVER STOP rising.