Issue type:

• 🐌 Performance issue

I was investigating high CPU usage on my server when no players are connected (and my server doesn't have any chunkloader mods, so everything should be idle when there's nobody around), and from some experimentation it looks like Integrated Dynamics is keeping the chunks where its networks exist loaded all the time. I'm still trying to come up with a definite isolated repro case and will update the bug with more detail if I manage it, but wanted to bring this up in case there's anything you're already aware of.

Things I've found so far:

1. Removing integrated dynamics from the modpack prevents the excess CPU usage - when nobody's connected, the server CPU usage drops from ~20% to <2%.
2. Destroying all the integrated dynamics objects built in the world while keeping the mod loaded also seems to work.
3. Running the VisualVM sampling profiler against the server VM while nobody is connected shows a lot of integrated dynamics calculation functions being called on an ongoing basis, but also a lot of activity from Thermal Dynamics trying to move things around in ducts. However, without ID being loaded, the duct activity stops, which suggests that the ducts are only consuming CPU when the chunks they are in are loaded by something else.
4. Starting a new world and just building a trivial ID network with a redstone reader connected to a redstone writer set to output a signal whenever the input reads <5 signal looks like it's enough to cause the ongoing activity, though the total CPU load here is much lower since there's nothing else in the chunk in question, and so even if it's keeping it loaded, it doesn't increase tick time anywhere near as much as in my actual save where the ID networks are in the middle of a large automated base.

Versions:

Exact versions of the following mods, not just latest.

• This mod: 0.10.1
• CyclopsCore: 0.11.1
• Minecraft: 1.12.2
• Forge: 14.23.1.2587

Are you sure your network is not just part of the spawn chunks? Those will always be chunkloaded.

Will look into it in any case.

I wrote a tiny mod to log all chunk load and unload events and dump stacktraces of what caused the chunk loads, and it looks like I have several mods that are causing chunks to be loaded when no players are online, and Integrated Dynamics appears to be one of them. The chunks get unloaded again after MC's usual cache expiry time and then immediately reloaded on the next tick, so it's definitely not part of the spawn chunks or an intentional chunkloader.

The stack I currently see that implicates Integrated Dynamics is:

java.lang.Throwable
	at com.example.examplemod.ExampleMod.chunkLoaded(ExampleMod.java:21)
	at net.minecraftforge.fml.common.eventhandler.ASMEventHandler_4_ExampleMod_chunkLoaded_Load.invoke(.dynamic)
	at net.minecraftforge.fml.common.eventhandler.ASMEventHandler.invoke(ASMEventHandler.java:90)
	at net.minecraftforge.fml.common.eventhandler.EventBus.post(EventBus.java:179)
	at net.minecraft.world.chunk.Chunk.onLoad(Chunk.java:920)
	at net.minecraftforge.common.chunkio.ChunkIOProvider.syncCallback(ChunkIOProvider.java:105)
	at net.minecraftforge.common.chunkio.ChunkIOExecutor.syncChunkLoad(ChunkIOExecutor.java:94)
	at net.minecraft.world.gen.ChunkProviderServer.loadChunk(ChunkProviderServer.java:130)
	at net.minecraft.world.gen.ChunkProviderServer.loadChunk(ChunkProviderServer.java:101)
	at net.minecraft.world.gen.ChunkProviderServer.provideChunk(ChunkProviderServer.java:147)
	at net.minecraft.world.World.getChunkFromChunkCoords(World.java:361)
	at net.minecraft.world.World.getChunkFromBlockCoords(World.java:353)
	at net.minecraft.world.World.getTileEntity(World.java:2597)
	at org.cyclops.cyclopscore.helper.TileHelpers.getSafeTile(TileHelpers.java:41)
	at org.cyclops.cyclopscore.helper.TileHelpers.getCapability(TileHelpers.java:115)
	at org.cyclops.cyclopscore.helper.TileHelpers.getCapability(TileHelpers.java:90)
	at org.cyclops.integrateddynamics.core.path.Cluster.fromNBT(Cluster.java:74)
	at org.cyclops.integrateddynamics.core.network.Network.fromNBTEffective(Network.java:204)
	at org.cyclops.integrateddynamics.core.network.Network.update(Network.java:376)
	at org.cyclops.integrateddynamics.core.TickHandler.onTick(TickHandler.java:55)
	at net.minecraftforge.fml.common.eventhandler.ASMEventHandler_262_TickHandler_onTick_TickEvent.invoke(.dynamic)
	at net.minecraftforge.fml.common.eventhandler.ASMEventHandler.invoke(ASMEventHandler.java:90)
	at net.minecraftforge.fml.common.eventhandler.EventBus.post(EventBus.java:179)
	at net.minecraftforge.fml.common.FMLCommonHandler.onPostServerTick(FMLCommonHandler.java:264)
	at net.minecraft.server.MinecraftServer.tick(MinecraftServer.java:785)
	at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:590)
	at java.lang.Thread.run(Thread.java:748)

I'm continuing to investigate the other mods that are implicated as well, and might actually polish this trivial mod up to be a proper debugging tool and release it..

Hmm, this line may be causing the problem:
org.cyclops.integrateddynamics.core.network.Network.fromNBTEffective(Network.java:204)

Thanks! This will definitely help!

I experimented with different combinations of the mods that are generating chunk loading events that I found in my log, and some of the mods are only causing "secondary" effects - loading additional chunks after an adjacent chunk is already loaded (e.g. Storage Drawers' drawer controller blocks: when the chunk containing the controller is loaded, its scan for connected drawers crosses chunk boundaries without checking if they are already loaded).

But, Integrated Dynamics doesn't appear to be one of those: I can remove all the other mods involved and ID will be left as the sole thing causing chunk loading events after the spawn chunks are loaded, and none of the ID networks on my server are near the spawn chunks.

Ok, just did some investigating.

When a world is loaded, ID will load all networks in that world to check if they are still valid. As ID doesn't have a concept of a central controller block, there's no (really efficient) way around this. After that, the chunks are free to unload again if needed.

I did some testing, and if chunks with networks inside them become unloaded by for example a player leaving them, they will become properly unloaded. They will not load again until a player enters the chunks according to my tests.

@tornewuff For your test cases, do the chunks with ID networks inside them stay loaded? If so, are there any special things in those chunks/networks that could be worth mentioning?

Aha - okay, this is interesting :)

Testing again, it looks like ID is actually behaving as you expect here: if I remove all the other mods causing chunkloading then ID only loads three chunks, once, and once they unload it doesn't touch them again. The problem is that when thermal dynamics is also loaded, the chunks that ID loads contain partial TD duct networks, and TD then loads four more chunks that are connected to the initial three via ducts. After a short while, the server unloads some but not all of these chunks from the cache, but the remaining loaded chunks still contain TD ducts, and so TD ends up reloading all of them again immediately.. the chunks were loaded at slightly different times since they were loaded for different reasons, so they don't all time out of the cache as a unit, and so the ping-ponging goes on forever.

So, it looks like I don't have a bug with ID after all - it's just the first thing to act and triggers bugs in other mods as a result.

Thanks for looking into this and confirming the expected behaviour; that was very helpful to figure out this complex situation. (on my real non-test server there are even more complex dependencies going on as there are also several other mods with bugs that cause additional "chunk x is loaded -> load chunk y" chains, which might also interact with each other causing more cycles).

When a world is loaded, ID will load all networks in that world to check if they are still valid.

Does this need to happen as soon as the world is loaded? Could the checking of each network be deferred until some chunk that it exists in gets loaded for some other reason?

@josephcsible Theoretically, we could indeed delay this until one of the network cables is loaded. Doing this in an efficient way would however be not so trivial, so I don't see this as as being essential.

Does this apply to all ID machines? Like for example, if someone was using the RF batteries or the automated squeezer/dying basins?

@nullKomplex Yep, it does apply to those as well. Load should be minimal though.

@farigiss ID does not do any chunkloading. Networks will only function if there is something else on the chunk that is loading that chunk.

I used the /chunkloaders command (from ChickenChunks, I think) and it reveals that any chunk with ID blocks in it loads chunks (marked red).
The /save-all command unloads all temporarily loaded chunks. All the chunks loaded by ID do indeed unload, only to come back half a second later.
It is not caused by actual chunkloaders (not as a block, nor FTBUtilities chunk loading, nor spawn chunks or player proximity), since there aren't any there. Removing the ID stuff from that chunk stopped it from being loaded.

This does not seem to be a proper chunkloader since there is no ticket associated with it. But it does load chunks and keep them loaded indefinitely, and loads them again after a server restart.

This is on Direwolf20 v2.4.0 for 1.12.2

I too have noticed that every chunk this mod is in (or borders on) gets chunkloaded.

While the performance hit by itself might be minimal, the chunkloading can have a huge impact.

All the unintentionally loaded chunks usually also load other stuff from other mods.

If you have a server that's been up for over a month, and people start going inactive, all those loaded chunks are bogging down the server and I can't just turn off the chunk loading like I can with a proper chunkloader. To stop the ID chunkloader, I'd have to destroy the person's base to look for every bit of cable.

I think that makes this a severe issue that will effectively ruin a server map over time.

@farigiss Could you try reproducing this without any other mods installed? Because there may be other mods influencing this behaviour. If you can reproduce it, be sure to open a separate bug report, I'll look into fixing it then.