Rubidium

Rubidium

34M Downloads

Constant memory leak while playing

MattB70 opened this issue ยท 23 comments

commented

The attached log shows a pretty much constant memory leak occurring while playing on the following versions:

Minecraft 1.18.2
Forge 40.1.54
Rubidium 0.5.2a
Oculus 1.2.5

Log:

Full latest log: https://gist.github.com/MattB70/6ecdf036fb9d0a90b693ff2257dc93d3
This pack contains many mods, including some that might not play nice with Sodium such as Chisels & Bits, Tinkers, Create, and more. While I expect some jank, this constant memory leak is definitely something more. Let me know if I can help by providing more info. I've also linked a video of it happening with BetterF3 mod and the FPS graph enabled. My suspicion is that it is caused by an interaction between Create and Sodium. This issue may be entirely related to Sodium and not the port itself, but requesting compat for a mod not available on Fabric seems a little odd to me. Not sure if I should create a unique issue there, I may open an issue stub and link to this here, but they usually just ignore issues related to ports, can't blame them.

Video:

This video demonstrates the results of a memory leak likely caused by Sodium's Forge port and is exasperated by the presence of Create. The buildings I am flying towards when the memory leak gets worse have Create mod blocks inside of them. This memory leak still happens everywhere else including fresh overworld chunks, but the leak is small enough that I do not feel its effects (though they are still there).
Video: https://www.youtube.com/watch?v=7W5JkxfjxIo

Update:

After further testing, the issue may actually be related to Chisels & Bits. As you get closer to Chisels & Bits blocks, they need to load again and are likely causing the bigger leak. The smaller constant leaking is still unknown.

Update 2:

The memory leak seems to increase with weather.

commented

This happens less often when not running the client and the server on the same machine, such as singleplayer with no external server running.

commented

Here is a snipped of a log with memory tracing enabled.
log: https://gist.github.com/MattB70/dfe31f53889d7b967fcd16bd202d360e

commented

does not happen for me.

commented

I also have a memory leak. I have a beefy computer so I don't feel the effects that much and can play for a good 4h but afterwards I get logged out of my personal server saying I ran out of memory and it exited to avoid corruption.

I suspect ferritecore or entityculling, maybe biome youll go since we all have those mods in common and they do things that could create a memory leak.

commented

Seems like the memory leak got significantly worse with ferritecore removed, so it's good to keep it as it's likely what's holding the game together.

commented

maybe something to be tried eventually, but since the issue takes around an hour to manifest, i don't look forward to the trial and error, especially since without any of the mods i wouldn't be able to join my server ๐Ÿ˜…

and for what it's worth, i'm the only one in the group of people who plays on this server that seems affected, and "coincidentally" i'm the only one with an AMD card (a 4GB RX480 with latest catalyst software, for reference), so...

commented

i assume this is also why i'm getting OutOfMemoryError crashes on chunk rendering after an hour or so of playing in my case...

Minecraft: 1.18.2
Forge: 40.2.10
Rubidium: 0.5.6
Oculus: 1.6.4
Java: OpenJDK 17.0.2

minimum and maximum memory are both set to 8G (out of a system RAM of 24 GB), i just updated my mods to the latest available versions with prismlauncher yesterday

the same pack with optifine in place of rubidium+oculus+extras+flywheel-compat does not cause an OOM crash but performs generally much worse

full mod list
Mods:
  [โœ”] Aquaculture-1.18.2-2.3.12
  [โœ”] AutoRegLib-1.7-53
  [โœ”] balm-3.2.6
  [โœ”] BiomesOPlenty-1.18.2-16.0.0.134
  [โœ”] blueprint-1.18.2-5.5.0
  [โœ”] Bookshelf-Forge-1.18.2-13.3.56
  [โœ”] BotanyPots-Forge-1.18.2-8.1.28
  [โœ”] buildersaddition-1.18.2-20220308a
  [โœ”] carryon-1.18.2-1.17.1.12
  [โœ”] cccbridge-mc1.18.2-forge-v1.5.1
  [โœ”] cc-tweaked-1.18.2-1.101.3
  [โœ”] cfm-7.0.0-pre35-1.18.2
  [โœ”] cgm-1.3.3-1.18.2
  [โœ”] Chimes-v2.0.1-1.18.2
  [โœ”] cloth-config-6.5.102-forge
  [โœ”] cofh_core-1.18.2-9.2.1.39
  [โœ”] comforts-forge-1.18.2-5.0.0.6
  [โœ”] Compat-O-Plenty-1.18.2-2.0.1
  [โœ”] connectedness-1.18.2-2.0.1a
  [โœ”] connectivity-1.18.2-3.2
  [โœ”] cookingforblockheads-forge-1.18.2-12.2.0
  [โœ”] corail_woodcutter-1.18.2-2.3.9
  [โœ”] create-1.18.2-0.5.1.c
  [โœ”] createaddition-1.18.2-20230623a
  [โœ”] crittersandcompanions-1.18.2-1.0.2
  [โœ”] Decorative Blocks Compat-forge-1.18.2-2.1.0
  [โœ”] Decorative Blocks-forge-1.18.2-2.1.2
  [โœ”] DiagonalFences-v3.2.3-1.18.2-Forge
  [โœ”] dynamiclightsreforged-1.18.2_v1.3.3
  [โœ”] DynamicTrees-1.18.2-1.0.4
  [โœ”] DynamicTreesBOP-1.18.2-3.0.1
  [โœ”] DynamicTreesPlus-1.18.2-1.0.4
  [โœ”] DynamicTreesQuark-1.18.2-2.3.0-Beta2
  [โœ”] ears-forge-1.18-1.4.5
  [โœ”] fairylights-5.0.0-1.18.2
  [โœ”] framework-forge-1.18.2-0.4.3
  [โœ”] fusion-1.0.5-forge-mc1.18
  [โœ”] geckolib-forge-1.18-3.0.57
  [โœ”] goprone-1.18.2-2.2.1
  [โœ”] grappling_hook_mod-1.18.2-v13
  [โœ”] HammerLib-1.18.2-18.2.16
  [โœ”] ImmersiveEngineering-1.18.2-8.4.0-161
  [โœ”] Jade-1.18.2-forge-5.2.6
  [โœ”] JadeAddons-1.18.2-forge-2.5.0
  [โœ”] jei-1.18.2-forge-10.2.1.1005
  [โœ”] jeiintegration_1.18.2-9.0.0.37
  [โœ”] jmi-forge-1.18.2-0.13-43
  [โœ”] journeymap-1.18.2-5.9.7-forge
  [โœ”] LibX-1.18.2-3.2.19
  [โœ”] macawsbridgesbop-1.18.2-1.4
  [โœ”] macawsroofsbop-1.18.2-1.5
  [โœ”] mcw-bridges-2.1.0-mc1.18.2forge
  [โœ”] mcw-doors-1.1.0forge-mc1.18.2
  [โœ”] mcw-fences-1.0.7-mc1.18.2forge
  [โœ”] mcwfencesbop-1.18.2-1.4
  [โœ”] mcw-furniture-3.1.0-mc1.18.2forge
  [โœ”] mcwfurnituresbop-1.18.2-1.2
  [โœ”] mcw-lights-1.0.6-mc1.18.2forge
  [โœ”] mcw-paths-1.0.3forge-mc1.18.2
  [โœ”] mcw-roofs-2.2.4-mc1.18.2forge
  [โœ”] mcw-trapdoors-1.1.1-mc1.18.2forge
  [โœ”] mcw-windows-2.1.1-mc1.18.2forge
  [โœ”] MoreVanillaLib-1.18.2-3.1.1
  [โœ”] MouseTweaks-forge-mc1.18-2.21
  [โœ”] mutil-1.18.2-4.5.0
  [โœ”] naturalist-forge-1.1.1-1.18.2
  [โœ”] NaturesCompass-1.18.2-1.9.7-forge
  [โœ˜] ocrenderfix_sodium-forge-1.1.0.jar (disabled)
  [โœ”] oculus-flywheel-compat-1.18.2-0.2.1
  [โœ”] oculus-mc1.18.2-1.6.4
  [โœ”] pamhc2crops-1.18.2-1.0.5
  [โœ”] pamhc2foodcore-1.18.2-1.0.3
  [โœ”] pamhc2foodextended-1.18.2-1.0.5
  [โœ”] pamhc2trees-1.18.2-1.0.4
  [โœ”] Ping-1.18-1.8.0
  [โœ”] polymorph-forge-1.18.2-0.49
  [โœ”] PuzzlesLib-v3.4.6-1.18.2-Forge
  [โœ”] Quark-3.2-358
  [โœ”] rechiseled-1.1.2-forge-mc1.18
  [โœ”] reforgium-1.18.2-1.0.12a
  [โœ”] rubidium_extras-1.18.2_v1.3.2
  [โœ”] rubidium-0.5.6
  [โœ”] SolarFluxReborn-1.18.2-18.2.4
  [โœ”] StorageDrawers-1.18.2-10.2.1
  [โœ”] supermartijn642configlib-1.1.7-forge-mc1.18
  [โœ”] supermartijn642corelib-1.1.12-forge-mc1.18
  [โœ”] Swashbucklers 2.5
  [โœ”] TerraBlender-forge-1.18.2-1.2.0.126
  [โœ”] tetra-1.18.2-4.10.1
  [โœ”] tetranomicon-1.4-1.18.2
  [โœ”] thermal_dynamics-1.18.2-9.2.1b.13
  [โœ”] thermal_expansion-1.18.2-9.2.0.20
  [โœ”] thermal_foundation-1.18.2-9.2.0.46
  [โœ”] thermal_integration-1.18.2-9.2.0.16
  [โœ”] towns_and_towers_forge-1.10.0.1+1.18.2
  [โœ”] VanillaExcavators-1.18.2-4.1.3
  [โœ”] VanillaHammers-1.18.2-4.1.2
  [โœ”] waystones-forge-1.18.2-10.2.1
  [โœ”] xptome-1.18.2-2.1.7
detailed crash
Exception in thread "Chunk Render Task Executor #4" Exception in thread "Chunk Render Task Executor #0" java.lang.OutOfMemoryError
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.nmemReallocChecked(MemoryUtil.java:484)
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.memRealloc(MemoryUtil.java:517)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.setBufferSize(VertexBufferBuilder.java:36)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.start(VertexBufferBuilder.java:88)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuildBuffers.init(ChunkBuildBuffers.java:62)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.tasks.ChunkRenderRebuildTask.performBuild(ChunkRenderRebuildTask.java:64)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder.processJob(ChunkBuilder.java:267)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder$WorkerRunnable.run(ChunkBuilder.java:309)
	at java.base/java.lang.Thread.run(Thread.java:833)
java.lang.OutOfMemoryError
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.nmemReallocChecked(MemoryUtil.java:484)
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.memRealloc(MemoryUtil.java:517)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.setBufferSize(VertexBufferBuilder.java:36)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.start(VertexBufferBuilder.java:88)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuildBuffers.init(ChunkBuildBuffers.java:62)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.tasks.ChunkRenderRebuildTask.performBuild(ChunkRenderRebuildTask.java:64)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder.processJob(ChunkBuilder.java:267)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder$WorkerRunnable.run(ChunkBuilder.java:309)
	at java.base/java.lang.Thread.run(Thread.java:833)
Exception in thread "Chunk Render Task Executor #2" java.lang.OutOfMemoryError
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.nmemReallocChecked(MemoryUtil.java:484)
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.memRealloc(MemoryUtil.java:517)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.setBufferSize(VertexBufferBuilder.java:36)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.start(VertexBufferBuilder.java:88)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuildBuffers.init(ChunkBuildBuffers.java:62)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.tasks.ChunkRenderRebuildTask.performBuild(ChunkRenderRebuildTask.java:64)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder.processJob(ChunkBuilder.java:267)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder$WorkerRunnable.run(ChunkBuilder.java:309)
	at java.base/java.lang.Thread.run(Thread.java:833)
Exception in thread "Chunk Render Task Executor #5" java.lang.OutOfMemoryError
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.nmemReallocChecked(MemoryUtil.java:484)
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.memRealloc(MemoryUtil.java:517)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.setBufferSize(VertexBufferBuilder.java:36)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.start(VertexBufferBuilder.java:88)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuildBuffers.init(ChunkBuildBuffers.java:62)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.tasks.ChunkRenderRebuildTask.performBuild(ChunkRenderRebuildTask.java:64)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder.processJob(ChunkBuilder.java:267)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder$WorkerRunnable.run(ChunkBuilder.java:309)
	at java.base/java.lang.Thread.run(Thread.java:833)
Exception in thread "Chunk Render Task Executor #3" java.lang.OutOfMemoryError
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.nmemReallocChecked(MemoryUtil.java:484)
	at MC-BOOTSTRAP/org.lwjgl/org.lwjgl.system.MemoryUtil.memRealloc(MemoryUtil.java:517)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.setBufferSize(VertexBufferBuilder.java:36)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.start(VertexBufferBuilder.java:88)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuildBuffers.init(ChunkBuildBuffers.java:62)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.tasks.ChunkRenderRebuildTask.performBuild(ChunkRenderRebuildTask.java:64)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder.processJob(ChunkBuilder.java:267)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder$WorkerRunnable.run(ChunkBuilder.java:309)
	at java.base/java.lang.Thread.run(Thread.java:833)
[21:08:47] [Chunk Render Task Executor #1/ERROR] [me.je.mo.so.cl.ut.NativeBuffer/]: EMERGENCY: Tried to allocate 1141760 bytes but the allocator reports failure
[21:08:47] [Chunk Render Task Executor #1/ERROR] [me.je.mo.so.cl.ut.NativeBuffer/]: EMERGENCY: ... Attempting to force a garbage collection cycle (attempt 1/3)
[21:08:47] [Chunk Render Task Executor #1/ERROR] [me.je.mo.so.cl.ut.NativeBuffer/]: EMERGENCY: Tried to allocate 1141760 bytes but the allocator reports failure
[21:08:47] [Chunk Render Task Executor #1/ERROR] [me.je.mo.so.cl.ut.NativeBuffer/]: EMERGENCY: ... Attempting to force a garbage collection cycle (attempt 2/3)
[21:08:47] [Chunk Render Task Executor #1/ERROR] [me.je.mo.so.cl.ut.NativeBuffer/]: EMERGENCY: Tried to allocate 1141760 bytes but the allocator reports failure
[21:08:47] [Chunk Render Task Executor #1/ERROR] [me.je.mo.so.cl.ut.NativeBuffer/]: EMERGENCY: ... Attempting to force a garbage collection cycle (attempt 3/3)
Exception in thread "Chunk Render Task Executor #1" java.lang.OutOfMemoryError: Couldn't allocate 1141760 bytes after 4 attempts
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.util.NativeBuffer.allocate(NativeBuffer.java:114)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.util.NativeBuffer.(NativeBuffer.java:30)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.util.NativeBuffer.copy(NativeBuffer.java:36)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.model.vertex.buffer.VertexBufferBuilder.pop(VertexBufferBuilder.java:104)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuildBuffers.createMesh(ChunkBuildBuffers.java:93)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.tasks.ChunkRenderRebuildTask.performBuild(ChunkRenderRebuildTask.java:172)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder.processJob(ChunkBuilder.java:267)
	at TRANSFORMER/[email protected]/me.jellysquid.mods.sodium.client.render.chunk.compile.ChunkBuilder$WorkerRunnable.run(ChunkBuilder.java:309)
	at java.base/java.lang.Thread.run(Thread.java:833)
commented

Yeah I understand. That's a very interesting coincidence. Just by curiosity, do you know if there is any other player that has a similar amount of VRAM on his GPU? If there is another player with a Nvidia GPU that has about the same amount of VRAM as you, that would support your theory. If everyone else has more VRAM, than it might just be because they have enough spare to power through the leak for multiple hours.

commented

so nobody goes down my own wild goose chase: i thought this might've been related to lwjgl. on version 3.2.3 they note:

jemalloc: Updated to 5.2.1 (up from 5.2.0)

and in jemalloc's changelog for 5.2.1:

This release is primarily about Windows. A critical virtual memory leak is resolved on all Windows platforms. The regression was present in all releases since 5.0.0.

however, even after selecting lwjgl version 3.3.1 through my launcher (the previous version available being 3.2.2, which still contains the regression), i still experience the same OOM issues after playing the game for an hour or so.

commented

The issue is back in the latest version, and I don't use lwjgl.

commented

i'm not aware of a way you can run minecraft without lwjgl

commented

Lol you're right. It's baked in. I'll run a few more tests tonight.

commented

But if lwjgl was the culprit, wouldn't most people playing Minecraft see that problem?

commented

that is true, i just thought maybe it was a combined effort which caused it, say the average player without the same load as rubidium plus all these mods wouldn't experience the same thing unless they were playing for several days...

on further inspection i don't think this is system RAM at all. the chunk allocator seems to use VRAM. i'm wondering if the Use Persistent Mapping option under chunk memory allocator for rubidium, plus the fact i have an AMD card, might be causing it

this is corroborated by the fact that whenever i get an OOM crash it usually takes other GPU-intensive applications out with it, like web browsers (including discord, which is a web browser with the serial numbers filed off)

commented

On my side, I simply measured the java application of Minecraft growing up to 18gb of used RAM.
Basically, the game thinks it uses its allocated memory but there is a leak and Windows can keep track of how much it gave Minecraft. So I can confirm you that on my end it was a typical memory leak.

commented

these might be two unrelated memory leak issues then

commented

There is a VRAM issue with last couple AMD GPU drivers. My original issue here was with an RTX2080 on AM4, but now I'm experiencing the same thing on a totally different system with an 7900xtx on AM5. I also know there is another issue with entities being rendered regardless of distance, but it's probably unrelated. I've got nothing more than that to report. It presents the same as the original issue.

commented

Since you have a relatively small modpack @Lana-chan, you could brute force test by removing half of them and checking if the issue still occurs. If it does, you know it is in the half tested, if doesn't, it is in the other half. You can keep repeating this process until you have isolated the culprit.

I also wonder if the renderer would resort to using RAM if it uses its full VRAM.
The issues may be unrelated but they are very similar so it leaves me wondering.

I also had issues with AMD previously but now Nvidia.
They may have been more noticeable with my AMD gpu but at the time my whole system was less powerful.

commented

the player with the lowest GPU has a 4GB 950M laptop, the other has a 6GB 1060, neither have any issues. AMD has been futzing with their opengl drivers more lately and i wouldn't be surprised if that has to do with it, too

EDIT: the player with the 950M finally hit the very same identical symptom with their discord tanking along with the same OOM errors, after a whopping 9 hour playtime ๐Ÿ˜

commented

Yeah so basically the issue is compounded with the shaky AMD drivers but its still there nonetheless. Thanks for all the info.

commented

I've been investigating this memory leak issue recently, for 1.16.5 and 1.18.2, and now I strongly suspect that the RenderLayer#getBlockLayers method is causing the problem.

Let's take a look at the it: Every time a mod calls it, one immutable list will be created, while I believe this performBuild method will be called many times during the game process. So, trillions of immutable lists are created and accumulating in memory, and finally the game becomes laggy or crashes.

Here I cache the list so it should be created for only one time now, can you try this build(This is for MC 1.18.2, if you're on 1.16.5, try this) to see whether or not the leak issue is resolved? Thanks. If it can't be resolved, then I'm out of options too, just pretend I didn't say anything.

For 1.19.2 or above, even though we don't know in which version Mojang implemented this, the list has already been cached.

commented
commented

Fixed in Dev version