Memory leak + unclosed thread
CammiePone opened this issue ยท 26 comments
Version Information
lithium-fabric-mc1.17.1-0.7.4
Expected Behavior
My game to not max out the 10 GB of allocated space within 30 minutes of just roaming the world, and for the JVM to close completely upon Alt + F4
Actual Behavior
RAM usage starts around 4.5 gigs and steadily climbs until it takes up the entire allocated amount within 30 minutes to an hour, depending on how much exploration happens.
Upon doing Alt + F4 while in a world, the JVM also refuses to completely close and runs in the background, steadily using more and more RAM until it caps out around 10 gigs. Requires Task Manager to kill it.
Reproduction Steps
- Load the game up and create a new world
- Log how much RAM is being used initially
- Run around constantly for around 30 minutes to an hour backtracking as little as possible
- Check how much RAM is being used now
Other Information
I noticed there was an issue with hoppers causing memory issues, and that could be the same problem, but there was no mention of the thread that refused to close, and RAM continues to be eaten up even after closing the game, so not 100% that's the same thing.
Mod list:
- [1.17] Easier Gunpowder 1.1.6
- AdditionalAdditions-2.1.3
- aether-1.17.1-1.5.0
- AmbientEnvironment-fabric-1.17.1-6.0.22
- anthropophagy-1.17-4
- anvil-crushing-recipes-1.1.2+MC1.17-1.17.1
- appleskin-fabric-mc-mc1.17-2.1.3
- AppleTreesRev-Fabric-1.17.1-v2.5
- arcanus-1.20
- architectury-2.6.35
- armor-visibility-3.0.0-fabric
- authme-1.17-1.5.0
- autoconfig1u-3.3.1
- axolotl-item-fix-1.1.2
- aylyth-1.17-ALPHA-2
- balm-fabric-1.3.0+0
- BeaconOverhaul-1.4.2+1.17
- betterbiomeblend-1.17.0-1.2.9-fabric
- BetterDungeons-Fabric-1.17-1.0.2
- BetterMineshafts-Fabric-1.17-1.0.1
- BetterStrongholds-Fabric-1.17-1.1.2
- bewitchment-1.17-12
- blockus-2.2.10+1.17.1
- BOMD-1.2.3-1.17.1
- Botania-1.17.1-420-FABRIC-SNAPSHOT
- caelus-fabric-0.0.17-1.17
- caracal-1.17-1.3.2
- Cardinal-Components-API-3.1.1
- carrier-1.8.0
- cauldron-fix-1.0.0c
- charm-fabric-1.17-3.3.2
- charmonium-fabric-1.17-3.3.0
- CHAS-1.6-Fabric
- cleancut-1.17-4.0-fabric
- clear-skies-mc117-1.5.70
- clickthrough-1.17.1-fabric0.36.1-0.4
- cloth-api-2.0.54
- cloth-config-5.0.38-fabric
- Clumps-fabric-1.17.1-7.0.10
- combattweaks-1.2
- connectiblechains-1.17.1-1.2.3
- Couplings-1.5.1+1.17
- creeperaiupdated-fabric-1.4.0
- ctm-fabric-0.5.1+1.17
- culinaire-2.1
- custom-stars-0.6+1.17
- customsplashscreen-1.1.0
- DangerousStoneCutter-1.17-1.5
- dark-waters-1.17.1-0.0.1
- darkness-mc117-1.6.74
- DarkPaintings-fabric-1.17.1-4.0.2
- dawn-2.0.6
- DefaultSettings-1.17.x-3.1.0
- DimensionalDoors-4.0.0+beta.3-26.07.2021
- dracomelette-1.17-2
- DungeonsArise-1.17.1-2.1.47-fabric-release
- dynamic-music-1.3.1
- ecotones-0.8.1
- EditSign-1.17.1-2.2.3
- enchantedbookresign-1.17.1-2.0
- enderscape-0.1.1
- enhancedcelestials-1.17.1-2.0.5
- EntityCulling-Fabric-1.3.3
- extended-armor-bars-1.1.0
- extraalchemy-fabric-1.17.1-1.6.1
- extraorigins-1.17-9
- FABRIC_1.17_chipped-1.1.1
- fabric-api-0.41.0+1.17
- fabric-chunkpregen-0.3.4
- fabric-language-kotlin-1.6.5+kotlin.1.5.31
- fabric-seasons-1.3.1-BETA+1.17
- fallingleaves-1.7.4+1.17.1
- fancymenu_fabric_2.3.5_MC_1.17-1.17.1
- fireworkfrenzy-1.6
- geckolib-fabric-1.17-3.0.19
- goldenhoppers-1.1.0
- grass-pass-1.17-1.0.0
- grind-enchantments-1.4.0+1.17.1
- grizzlybear-1.17-1.2.0
- haema-1.7.7+1.17
- hookshot-r20
- horse-stats-vanilla-4.1.8
- icarus-1.8
- IKnowWhatImDoing-3.1.0-fabric
- illuminations-1.9
- immersive-portals-0.29-mc1.17.1-fabric
- impaled-1.0.1
- impersonate-2.3.5
- incantationem-1.1.2+1.17.1
- interactic-0.1.6-1.17
- iris-and-sodium-mc1.17-1.1.2+build.9
- item-model-fix-1.0.2+1.17
- keybind-search-1.17-1.0.0
- konkrete_fabric_1.3.0_MC_1.17-1.17.1
- krypton-0.1.4
- lacrimis-2.1.3+1.17.1
- Lakeside-1.1.0+1.17
- lambdynamiclights-2.0.2+1.17
- lazydfu-0.1.2
- light-fix-1.0.0
- light-overlay-6.0.1-fabric
- lithium-fabric-mc1.17.1-0.7.4
- lock_minecart_view-for-MC1.17-rc1-1.2.1
- luminous-wool-1.1.1
- magitekmechs-fabric-MC1.17-1.0.4
- mavm-1.0.5
- Mesh-0.15.0-alpha
- mixintrace-1.1.0+1.17
- mobs-attempt-parkour-0.2.3-1.17-pre1
- modmenu-2.0.14
- mods-command-mc1.17.1-1.0.4
- monsters-in-the-closet-1.0.2+1.17
- morebannerfeatures-1.0.7
- morevillagers-FABRIC-1.17-2.1.1
- mostructures-1.3.0-pre1-1.17.1-fabric
- music-duration-reducer-1.2.1
- mythic-mounts-1.17.1-2.0
- nears-1.17-1.1.2
- no-potion-offset-1.0.2
- notenoughanimations-1.3.1
- notramplefabric-1.0.0
- observable-0.1.2-1.17-fabric
- onsoulfire-1.17-3
- OpenLoader-fabric-1.17.1-3.0.1
- Origins-1.17-1.1.2
- Origins-Classes-1.17-1.2.2
- outvoted-1.5.0-b1
- pal-1.3.0
- Patchouli-1.17.1-56-FABRIC
- Paxi-Fabric-1.17-1.2.2
- Pehkui-2.5.3+1.14.4-1.18
- piglessportals-1.1.0
- quickspyglasser-0.3.0
- ratsmischief-1.2.3
- realisticfirespread-1.17-3
- repairable-tridents-2.0.1
- repurposed_structures_fabric-2.4.3+1.17.1
- requiem-2.0.0-alpha.12
- requiem-pandemonium-2.0.0-alpha.12
- reroll-1.3.0-1.17.1
- respawnablepets-1.17-5
- right-click-clear-2.0.3
- riverredux-0.2.0
- RoughlyEnoughItems-6.0.307-alpha
- scorch-1.1.2
- sihywtcamd-1.5.2+1.17.1
- simple-gravestones-0.5
- smoothboot-fabric-1.16.5-1.6.0
- snow-under-trees-fabric-mc1.17.0-1.0.0
- somnus-fabric-0.0.14-1.17.1
- soundphysics-2.1.0
- spark-fabric-1.6.2
- starlight-1.0.0+fabric.73f6d37
- statues-1.0.1
- stoneholm-1.3
- stonevaults-1.1.0
- sync-mc1.17.x-1.2
- TaxFreeLevels-1.2.1
- tdnf-mc117-2.1.144
- The_Graveyard_(FABRIC)_1.4_for_1.17-1.17.1
- things-0.2.6
- torohealth-1.17.1-fabric-12
- TRansliterationLib-fabric-1.2.2-1.17.1
- traverse-4.0.0
- trinkets-3.0.4
- urns-mc1.17-1.0.1
- vehiclefix-1.1
- villager-hats-mod-1.2.1
- villagernames-1.17.1-2.0.0
- vistas-1.4.4
- voidtotem-fabric-1.17.0-1.0.0
- waystones-fabric-8.1.1+0
- whyamionfire-1.1.1
- wraith-harvestscythes-2.0.6
- YungsApi-1.17-Fabric-19
Can you test this with the latest 1.17.x/dev build? There has been a hopper related memory leak in 0.7.4
This was brought up in the Discord, asking if that fixed the issue, and it hadn't. I can test with the latest latest build when I get home just to be absolutely certain something hasn't changed.
Can you please check if this happens with just lithium installed and no other mods? If it doesn't, remove half your mods at a time to try and narrow down which specific conflict is causing issues.
It does indeed happen with just Lithium. Doubled RAM usage from 2.5 gigabytes to 5 gigabytes after just 7 minutes
Are you seeing something more significant than just normal GC behaviour? The Java heap will naturally increase in size gradually, and OS memory monitoring will not properly pick up garbage collections since the heap size doesn't change.
How exactly are you monitoring memory usage? And what have you set the max heap size (-Xmx) to?
-Xmx is set to 10 gigs.
I'm monitoring with Task Manager (yeah ik it's not the best, but the results are consistent) where only with Lithium do I see a dramatic increase in RAM usage over a period of time only while loading chunks. Without Lithium it never goes higher than 5.7 GB of usage and if I stop moving, goes back down to around 3 GB, whereas with Lithium it peaks at 10 GB and never goes back down.
If I have Immersive Portals installed, it tells me that there's not enough RAM allocated at the same time that Task Manager is getting around 10 GB of usage.
It's hard to tell what's going on without more information. Would you be able to join the Caffeine Discord so we can try out some troubleshooting and profiling steps to narrow down the cause?
Tested with just Lithium three times now, yes it does happen with just Lithium installed
I assume this does not happen with lithium alone, as you seem to be the first user reporting this problem while also having a long list of mods installed. I suggest you try to reproduce the issue with half of the mods present at a time to narrow it down to a small set of mods that together cause the memory leak.
Can you also test without lithium (vanilla or fabric loader only)? According to some users vanilla has issues at 8+GB of Ram
I am running into the same memory leak problem. I am using chunky to pregenerate a server world and with fabric installed it generates about 400 chunks the screeches to a halt. After a little while longer minecraft exits with an out of memory error, I run it with 6gb of ram and when I remove lithium it runs smoothly.
I have already tested a build with the fix for #283, it didn't fix this issue.
I think this might be related to the other JVM flags you are using but you haven't posted. Anyways this issue does not seem to be lithium related. If there was a memory leak, the game would eventually crash.
My guess would be that you messed with -XX:MaxHeapFreeRatio or -XX:MinHeapFreeRatio or similar flags depending on your JVM
https://www.oracle.com/java/technologies/javase/vmoptions-jsp.html
It looked like I could reproduce it, but actually, I can't. Maybe I also don't understand what the problem is
The issue report complains about the JVM/Game using memory, which is intended behavior. If you want the JVM to use less memory, give it less memory in its launch parameters. Be aware that Task Managers may not show the amount of physical memory that is in use, but some other value. Do you actually experience any problems that are related to the memory usage of the game?
However the JVM not correctly closing is an issue. But I cannot reproduce that
The issue with the JVM not closing is really inconsistent in getting to reproduce now, so Im wondering if another mod was actually the cause of that, and it has been fixed. I havent had much time to test it though, since I have been away for the past week.
The memory issue, however, persists. What is happening is that RAM usage increases the more chunks are loaded. This only happens with Lithium installed, whether it be my 180 mod modpack or just Lithium. No, I do not mean loading chunks and leaving them unloaded, nor do I mean changing your render distance to load more chunks at once. I just mean flying in a straight line non-stop.
With Fabric Loader on its own, the memory issue doesn't happen. With my 180 mod modpack without Lithium, the memory issue doesnt happen.
Can you try changing your JVM parameter -Xmx to a lower max heap size?
To me it is unclear why you think it is an issue that the game uses the memory you allow it to use.
In the case of flying forward, it is possible that lithium allows loading chunks faster, thus more chunks being in memory at the same time (assuming that unloading takes some time). This can lead to a higher total memory usage in comparison, especially when the JVM realizes that garbage collection is not necessary due to the remaining available space
It's at 10 gigs and someone else just said it happens at 6 gigs. I am still on the road so no, I cannot test it right now. Any lower than 6 gigs anyways and it'll use up all the RAM the modpack without Lithium uses while loading chunks (5.5 gigs).
The issue is it never stops using the RAM. It'll go up to 10, and then never go down. It idles at 4 gigs.
Let me just reiterate, for the who knows how many times, given I had to run multiple tests for people on here and in the Discord:
The modpack without Lithium caps out at 7.5 gigs of RAM usage when loading new chunks and goes back down to the 3-4 gigs it uses at idle if you stop moving.
With Lithium it will use the entire 10 gigs of RAM, causing other mods to no longer be able to write anything to memory, such as Immersive Portals. It stays there if you stop moving.
In both tests the game remains unpaused. This happens with or without other mods, the only difference is the amount of RAM used by the game without Lithium. With Lithium the numbers are the same, it just takes more time to use all the available memory.
The only JVM flag I messed with is -Xmx and enabling mixin debug so I can see when a mod breaks something. I have not pushed the game far enough to crash it because I usually close the game. However, @HughSparks has already stated the game DOES crash eventually due to Lithium.
I should also mention I had a crash pregenning my server, but it may have been Ecotones' fault. I never looked further into that crash after it happened because it was before I realised there was a leak in singleplayer with Lithium.
Just for posterity's sake, I loaded up my laptop on a mobile hotspot just to send you my JVM arguments.
-Dfml.ignorePatchDiscrepancies=true -Dfml.ignoreInvalidMinecraftCertificates=true -XX:HeapDumpPath=MojangTricksIntelDriversForPerformance_javaw.exe_minecraft.exe.heapdump -Xms256m -Xmx10G -Dmixin.debug.export=true
So unless GDLauncher adds other changes without listing them, these are all of them.