Multi Mine

Multi Mine

16M Downloads

Slowdown due to loadPosFile

clienthax opened this issue ยท 18 comments

commented

After pregening a large world i have around 4.3k tower entries in this file.

[21:56:39] [Server thread/INFO] [STDOUT]: [atomicstryker.battletowers.common.WorldGenHandler:loadPosFile:357]: Battletower Positions reloaded. Lines 4361, entries 4348

It seems that every time it loads the main thread locks up for a few seconds.

commented

As far i recall, this should only happen once on savegame load. And there is little i can do to speed up a brute force read of a text file ... was there a specific issue?

commented

Ive seen it load the file multiple times in a row,
It seems to load twice for each world load but ive seen it reload 4 times in a row before.

[03:21:58] [Server thread/INFO] [griefprevention]: [DIM7] 1 total claims loaded.
[03:21:58] [Server thread/INFO] [STDOUT]: [atomicstryker.battletowers.common.WorldGenHandler:loadPosFile:357]: Battletower Positions reloaded. Lines 4361, entries 4348
[03:21:58] [Server thread/INFO] [STDOUT]: [atomicstryker.battletowers.common.WorldGenHandler:loadPosFile:357]: Battletower Positions reloaded. Lines 4361, entries 4348

However the entries/lines are always the same, is it loading the same file for every world load, or are these meant to be different per world?

commented

BattleTowers-1.12.2.zip

Please try this file. I have rewritten the pos file handling a little - not the loading itself but the determination when to reload it

commented

@AtomicStryker Is this versioned the same as BattleTowers-1.12.2_1.6.4.jar ?, Need to know as running in a pack.

commented

I don't know. Check inside that file for the changelog.txt? Besides, what i posted is for you to try out, not to pack into any modpacks...

commented

If i encounter the issue in a modpack i am hosting a server for, i am obviously going to try the fix on said server where the issue was encountered..., Will check shortly

commented

@AtomicStryker I am on the same modpack as clienthax, same issue, and using your current version you uploaded here. No changes. Once the mod starts loading the positions file over and over the TPS of the server crashes to the bottom of the sea. Whenever the positions reload messages stop for a little bit the TPS fully recovers.

Im getting around 20-50 reloads before they eventually stop every few minutes.

commented

@clienthax @Zilacon This may be related to whatever sponge/bukkit system your using. I have been playing with the default server/forge setup and its only loaded battletowers once in 2 hours of play. Once I play again I will confirm this, otherwise no idea what to tell you.

commented

Il look into it from the sponge side.

commented

@Slayer5934 Were you switching worlds over and over? Remember on a public server with dozens of players everyone's switching worlds every minute.

commented

Servers being used on: Sponge & Bukkit servers. Confirmed on both.
How to reproduce: Load server with mod, join a world, enjoy the spam.
Modpack: Tekxit 3 Official 1.12.2 on Technic (0.94)
Logs:

[00:14:12] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:12] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:12] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:12] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:12] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:13] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:13] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:13] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:13] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:13] [Server thread/WARN]: Tried to over-fill a container
[00:14:13] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:14] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:14] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194
[00:14:14] [Server thread/INFO]: Battletower Positions reloaded. Lines 7207, entries 7194

Errors: No errors found for this problem, just the above logs.

The position file will load more and more depending on how many threads the world/chunk provider is given. On my server it is set to have 8 threads. So every world switch or join will reload the positions file 8 times. And it stacks. So if 4 players switched from Survival to Nether it will reload 32 times.

For clienthax it only shows as 2 times per world switch because he is using sponge and on a single thread, but is using a intel cpu that has 2 threads per core so its doing it twice per world switch for him. (i believe thats how that works).

Every dimension the server has a positions file for also multiplies the reload.

commented

I did some testing on Tekxit 3 1.12.2 with a default forge server, it does still happen on there just not all the time. Sometimes it loads once like its suppose to, sometimes it does twice for some reason, other times it doesnt even load the file while switching worlds (seems there is already some kind of check in place?) since I did this test on a local server with me being the only player I cant really tell if its causing any lag though or if it stacks but I have a feeling it does stack.

Is it possible to add a check to make sure it hasn't already been loaded? Or does it need to be loaded per thread always?

If there is any testing you need me to do just let me know, I'm sure clienthax and zilacon are probably willing to be testers as well since their servers are already effected :P

Edit: After a few more times switching worlds back and forth it doesn't seem to be reloading the tower positions anymore, which I think is the intended design but I'm not sure.

commented

I believe the tower positions file should be one large file in the root world directory (overworld dim0) with separate lists per dimension in one file and loaded into memory on start up and saved only when a new battletower is generated/destroyed.

commented

I think i see the issue here @AtomicStryker The tower Positions are a static field, yet for EVERY world load event you are loading the file, this is likely resulting in the same positions being in the position file of each world.

Im guessing you either need to make the towerpositions per world/dim name, or to only load the file globally one time.

commented

Basically what i said lol.

commented

Oh i see, unlike Ruins i didn't make a per-world handle system ... or rather, Ruins got that at some point, and i never made one for Battletowers.

commented

27817c9 in a short test it didnt crash immediatly, so i guess you can try Battletowers 1.6.5

commented

@AtomicStryker Can i get a compiled jar with those changes with the version set as 1.6.4? So its compatible with the modpack were hosting.