Lootr (Forge & NeoForge)

Lootr (Forge & NeoForge)

66M Downloads

[1.12] 0.3.1 Server Stall Crash

Krutoy242 opened this issue ยท 23 comments

commented

After updating 0.3.0 => 0.3.1, loading old world on dedicated server causing extremely low TPS (about 20000 ms for 1 tick) and eventually, crash (tick time treshold).

On 0.3.1, before i join the world i made Spark profile, because without players server run 2 TPS https://spark.lucko.me/49r1skzRXG
After i join i got crash after few minutes.

Downgrading to 0.3.0 and loading world from pre-update backup fixed issue.

debug.log with crash info
Crash Log


Also

This may be related to Lootr, but in another version of my modpack (on lootr 0.3.0 or even earlier), we had issue with carts x50 times copied on same position. This happen only once, so we didnt make an issue.

It may be unrelated, but this time i tried to call /cofh killall cart, and then server print 16831 same errors:

[Server thread/WARN] [net.minecraft.world.storage.loot.LootTable]: Tried to over-fill a container
commented

Yes, this is E2E-E server. There was one player that played for a week on this version. But only after week server started to stall

commented

Ok, i played some time again. Still stalls.
As before, removing Lootr fix stall issue.
lootr-1.12.2-0.3.2-BETA-2

debug with stall crash
crash-2022-07-19_19.32.05-server.txt

commented

@Krutoy242 Hmm... not sure why it would still happen. Is this an E2E-E server you're testing on?

commented

@Krutoy242 Can you give the beta I linked here a try on your server using the backup and let me know if the behavior is any different?

commented

give the beta a try

Sorry, i spent all my free time today on debugging already ๐Ÿ˜ฅ

So maybe in week or two.

commented

More info:

The stall and crash even happen on 0.3.0.

I just ran /cofh killall minecart and removed about 20,000 Lootr's Minecarts ๐Ÿ˜จ

I made profiler with it (1 tick about 30000 ms) https://spark.lucko.me/5BJxm2CRGg

commented

I've tweaked the minecart spawning to try and avoid this duplication; that is the reason for the lag. (This also includes the changes from the beta I linked above.)

https://cdn.discordapp.com/attachments/994620334399557635/995328553178644570/lootr-1.12.2-0.3.2-BETA-2.jar

Are you running a normal Forge server, or is it Sponge/one of these other hybrids?

commented

normal Forge server

Im running normal server.

tweaked the minecart spawning

How i can test this? The problem happened after a week of play or so.
Do you know how to reproduce the issue?

commented

No, I haven't been able to reproduce it, but there was some logic inherited from Lootr 1.16 that could potentially try to spawn the same minecart more than once if chunks were unloaded/loaded a certain way. I don't think this is necessary in 1.12 so I removed it. My best suggestion would be to simply play for another week and see if the problem is repeated.

commented

@Krutoy242 In the meantime, 0.3.2 has been released on CurseForge; I would expect it to fix your problem unless I've missed something.

commented

Let's try that and see if it works.

Ok, seems like 0.4.2-BETA-1 works fine. I ran server for 10 hours without players and no new Lootr Minecarts was spawned.
I could test further and play some time on server, but this could take a week.

Does -BETA-1 changes included in 0.4.4 ?

commented

Yes, they should be.

commented

clearing out entities before joining the server

Server stall even without players. Its happen after ~10 minutes of server running. Its 3-2 TPS first, then 1 TPS then stall.
Yes, clearing all entities before joining fix stall too. Last time i used /cofh killall *

using /forge entity list to see how many minecarts there

Ok, i will try to use this command next time.

commented

Unless new chunks are generating while there are no players online, there should be no Minecart entities being added to the world...

commented

Have you tried clearing out entities before joining the server, and using /forge entity list to see how many minecarts there are? It's possible that an excess were in unloaded chunks.

commented

Based on this new information, my next theory is that Entity#isAddedToWorld behaves differently in 1.12 and somehow it tries to spawn the same minecart numerous times. I'll try looking into this next.

It's strange that, to my knowledge, you are the only person to have this problem so far.

commented

Ok as i said, i removed Lootr to fix stall crash.
Today i installed it back, restarted server and run server for 4 hours. No one joined after restart.
And there is already 600 minecarts, lol

image

Update:

And its growing! After 10 minutes its +26 carts
image

Update 2:

Maybe this information unrelevant, but on E2E-E mineshafts generated with YUNG's Better Mineshafts.

commented

This error originally was reported by other E2E-E player on another server. He had all 50 minecarts on very same spot.
I will try this /tp command later.

image

commented

I believe in 1.12.2 you can still use the teleport trick to work out where Minecarts are: try /tp @e[type=lootr:lootr_minecart] ~ ~ ~ and it should print out a list of all the locations of the Minecarts. I'm curious if they're all in or near the same location, or if they're spread out over the map?

If they're all around the same location it could be that a single minecart is getting stuck in the entity ticker and never being cleared... (presuming 1.12's entity ticker uses a similar method of replacement).

Admittedly, as I don't believe there is asynchronous world generation in 1.12, it might be possible to just replace the minecarts at execution rather than deferring them. @embeddedt, thoughts?

commented

Admittedly, as I don't believe there is asynchronous world generation in 1.12, it might be possible to just replace the minecarts at execution rather than deferring them. @embeddedt, thoughts?

That's correct. Let's try that and see if it works.

https://cdn.discordapp.com/attachments/994620334399557635/1000072767552634950/lootr-1.12.2-0.4.2-BETA-1.jar

commented

There is strange thing i encountered.
Server was ran long time, and i came back to see 36000 minecarts. I ran /cofh killall lootr_minecart, and instead of just remove entities as /cofh killall usual do, it spawned tons of items. Next call of /forge entities list show about 1,000,000 of minecraft:item

commented

I wanted to bump this issue as I experienced a similar problem on my server, where I also had Yung's Mineshafts installed. I think, as mentioned above, that with Yung's mods, the server just continually generates chests in their locations and eventually times out the game.

For additional info, this server is in 1.19 and fabric.

Please start a separate issue for this; this issue is for 1.12.2.

commented

I wanted to bump this issue as I experienced a similar problem on my server, where I also had Yung's Mineshafts installed. I think, as mentioned above, that with Yung's mods, the server just continually generates chests in their locations and eventually times out the game.

For additional info, this server is in 1.19 and fabric.