Issue type:

• 🐛 Bug

Short description:

Shortly after starting the server stalls and crashes. This happened after we updated our server from SF4 v4.1.0 to v4.2.0. We kept the world and would like to keep it if possible.

Steps to reproduce the problem:

Starting the server...

1. ...

Expected behavior:

Not crash.

Versions:

• This mod: IntegratedDynamics-1.12.2-1.1.6
• Minecraft: 1.12.2 Sky Factory 4 v4.2.0
• Forge: forge-1.12.2-14.23.5.2854

Logfile:

Stall and Crash reports...
crash-2020-08-04_04.57.24-server.txt
stall-2020-08-04_04.57.09.txt

ServerHangWatchdog detected that a single server tick took 60.00 seconds (should be max 0.05)

Seems like a server load issue. Are you sure your server is powerful enough (CPU/RAM)?

We have 17.9 gb and a Ryzon 9 3900x 12 core. Should be plenty of power.

It did roughly the same thing when we updated to sf4 v4.2.2. It was fine before we added sponge. After we added sponge and all of our plugins it stalled and tps was less than 1. Never crashed.

We backtracked to sf4 4.2.0 using a factory reset option the server host has. Right now we're running on a new world completely fine, perfect 20 tps. But when we loaded the old world it stalled and crashed as seen above.

You reported this before in #900. Did the config change I suggested fix the problem?

The #900 error was solved by disabling "ingredientNetworkObserverEnableMultithreading". The server ran fine after we changed that option before we updated the server to sf4 4.2.0. Apologies for not informing you.

When I updated the server I preemptively disabled "ingredientNetworkObserverEnableMultithreading" to avoid having that issue with the new update. I enabled it again and the server performed better (4 tps) but still stalled. Here's the stall report it generated:
stall-2020-08-06_19.19.17.txt

Do you by any chance have a massive network somewhere? With a large number of item interfaces? That's the only explanation I can think of.

In any case, sharing your profiling output would also help to resolve this issue: https://github.com/CyclopsMC/CyclopsCore/blob/master-1.12/.github/CONTRIBUTING.md#performance-issues-snail

GitHub

CyclopsMC/CyclopsCore

Minecraft library mod for EvilCraft, Integrated Dynamics and others. - CyclopsMC/CyclopsCore

The server crashes within 60 seconds after loading. Its tricky to get a profile before the crash. I can give it some more shots if you need.

Ok, I set max-tick-time=-1 in server.properties to prevent Watchdog from crashing the server. No change in performance. Gathered much more on the profile this time! Let me know if you need even more.

hope this is enough....zip

Thanks, that's exactly what I needed!

Note to self: possible insertion via item interface into a massive network. Do some optimizations there:

Is there any way to stop/disable this network? I am not able to get into the server to break it.

You could enable safeMode in the ID config file, which should disable all network processing until it is disabled again. After enabling it, you could try to look for the problematic network and dismantle it manually. (It has something to do with a bonsai hopper)

I'm not aware of any large networks. I know of one that's and is set up similar to this:

It's a massive world with many islands. There are for sure more networks that I'm not aware of. If there's any file I can share from the world folder that might give you more information on any networks I'll be happy to share them in private if possible.

As for the profile, I managed to generate one right before it crashed.
OldWorldProfile872020.zip

Thanks for the profiler output.
I assume it did not run very long before the crash (?), as I don't see enough information in the output to start my search.

Enabling safeMode brought the server back to life!

We found the network after a while. I somehow managed to corrupt all the networks in the server. (Probably because I tried disabling/enabling safeMode in-game) They were fixed by replacing the variable block. The network in question had around 400 cables connected to Bonsai Pots in a compact storage dimension. I have the world still if you want to take a look at the network.

Thanks for all the help!

Great to hear the problem is solved.

Will keep this open for myself to look into optimization.