Loading cache takes a huge amount of RAM
Nensec opened this issue ยท 4 comments
Encounterd this while trying to troubleshoot an issue in the Enigmatica 2 modpack
EnigmaticaModpacks/Enigmatica2#289 (comment)
When the mod wants to load all structures of the overworld in a cache it results in a massive CPU bogged lagspike and the memory usage going to crazy amounts. In my case from 5gb used to 11gb used in the span of half a minute. In a previous instance where the server was configured to be only 8gb max it actually caused the entire server to hang.
[00:36:44] [Server thread/INFO] [Astral Sorcery]: Load CachedWorldData 'structures' for world 0
[00:36:45] [Server thread/INFO] [Astral Sorcery]: Loading of 'structures' for world 0 finished.
[00:36:45] [Server thread/WARN] [minecraft/MinecraftServer]: Can't keep up! Did the system time change, or is the server overloaded? Running 16293ms behind, skipping 325 tick(s)
[00:37:41] [Server thread/WARN] [minecraft/MinecraftServer]: Can't keep up! Did the system time change, or is the server overloaded? Running 17440ms behind, skipping 348 tick(s)
Our overworld has 450 chunks pre-genned, perhaps it is loading strctures that players have not even been to yet physically?
6 GB of memory taken in 450 chunks of structure caching would equate to 13+ megabytes of data per chunk. That's not possible, based in size of the structure data, unless there is something truly insane going on in interaction.
Either way, 1.10.11 is changing how this is called so this is largely irrelevant.
Out of curiosity, are you running Sponge?
Nope, just regular Forge that is part of the Enigmatica 2 modpack.
It could very well be that something else is causing the memory increase and CPU usage, this mod simply was the only thing that output anything in the log at the time.
6GB only for structure loading? You know all that's saved & cached per structure are 12 byte.
That would mean the mod has generated 500 MILLION structures.
And that is the lower end of things, the file would - due to the way NBT as datastructure is saved - most likely be larger than 6GB. - edit: eeh... compression is a thing i guess..
500 Million structures in 450 chunks? you can BARELY fit them there. every 4th block in that would would basically be a structure. Unlikely to say the least.
I am running 1.10.11 and I am still having this issue and don't have retro gen on or Thaumic JEI installed on the server, though I did pre-gen my world to a 6000 block radius. After loading the cache, my server's memory usage went up by 5 gigabytes.
Log: https://pastebin.com/LUxcVqZd