Common sense Info

• I play...
  • With a large modpack
  • Only with CQR and it's dependencies
• The issue occurs in...
  • Singleplayer
  • Multiplayer
• I have searched for this or a similar issue before reporting and it was either (1) not previously reported, or (2) previously fixed and I'm having the same problem.
• I am using the latest version of the mod (all versions can be found on github under releases)
• I read through the FAQ and i could not find something helpful (FAQ)
• I reproduced the bug without any other mod's except forge, cqr and it's dependencies
• The game crashes because of this bug

Versions
Chocolate Quest Repoured: Latest
Forge: 2854
Minecraft: 1.12.2

Describe the bug
Generate world without protections. CQR generates protection_region files anyways in the thousands which are continuously loaded and written to on every world save causing a massive memory leak. A world without these files or CQR data on a modpack will boot using 1GB of ram, with these files over 6-8GB on boot up. And the ram keeps increasing until it runs out of memory and deadlocks/crashes.

When updating the mod, it spams that all of the files are from an older version, even when the conversion from old to new versions is enabled in the config. Combined that with the above problem it is impossible to use a world generated or created on a newer version of CQR.

I have seen CQR itself use up to 28GB of ram on my server, no matter how much memory you give it, it will use it all up.
Every world save takes 30-60+ seconds, and in that time nothing ticks, everything's deadlocked.

This has been confirmed to be a reproducible issue by multiple server owners/people i know.

To Reproduce
Steps to reproduce the behavior:
This error is produced on the Tekxit 3.14 (PI) modpack by Slayer5934

1. Create world and pregenerate to spawn many dungeons.
2. Watch as the world size generated increases, the ram required to run the server sky rockets.
3. Update CQR
4. Your console now has no purpose, it only passes CQR errors. And maybe butter.
5. Enable conversion of old to new files in config, the version errors dont vanish until the structures.dat file is deleted, the entire CQR folder in the world folder needs to be purged to fix the errors.
6. Every world save will take 60 seconds or longer, basically deadlocking the server AND client (world was moved to single player, same issues occured).

Expected behavior
Not to generate protection_region files if protection is disabled in config.
Not to continuously load files into memory when the world is saved.
Not to cause an infinite memory leak.
Not to cause world saves to take longer then 1 second. (On modpacks with over 300 mods and a 25000x25000 world a world save takes less than a second).

Ironically i can't reprocude this issue with just CQR which is the root of all evil according to you. For me it works normally and doesn't cause saving times of 60 seconds.

I suggest you to do the following:

1. Actually follow the update instructions, this means deleting the cqrepoured config and the CQR folder before installing the update.
2. Be nicer towards our work
3. Disable structure file caching

Also a site note: The protected regions files only really mark a dungeon region. The pure existence of such a file DOES NOT MEAN that certain areas are protected, that are different things.

And honestly, i really doubt that cqr alone uses 32GB of RAM, more like the server's JVM uses all that which is normal java behavior.

Afaik the "conversion" feature doesn't really convert the files if i am informed correctly cause the responsible dev was strictly against including support for older files for whatever reason.

I was simply reporting my issue, in the format requested. Im not sure what in my post offended you or insulted you, but that was not my intention. My intention was simply to report my problem to you.

FYI, the "update old files" option only affects the structure files, not all the other files we use

I was offended of the phrasing of some sentences, yes that was unproffesional, though we do this as a hobby so expecting "professional" issue handling is a bit too much imo, but that is off topic.

Anyway, if you don't like us, we don't force you to keep the mod, you are free to un-install it.

Regarding your issue: I was not able to reproduce it, what i did was pregenerating a world with the size of 100k x 100k blocks and i did not experience what you reported.

You checked that the game crashed, do you happen to have a crash report?
If you don't: Please try the following:

1. Get a fresh installation, meaning no pre-setup configs or worlds, etc.
2. Boot up the game, then shut it down.
3. Enter CQR's config and disable"cacheStructureFiles"
4. Try to reproduce your issue by pregenerating a world with the size of 25k x 25k (which should be the size you reporteed)

Site note: We are doing this for free, we do not get any revenue from making this mod. In fact the only ones that get revenue from our work are modpack creators running servers and cashing in donations, etc. So in fact one in general doesn't have the right to expect that we spend 24h/7d of our available time to work on their issues. This is just a site note and i am mentioning it only cause i sadly feel it is necessary here, though i could be wrong with that assumption.

Also a question: What would be professional according to you?
(Yes, changing the title was unprofessional and not necessary but the title you gave imo sounded a lot like blackening us cause it contained words such as deadlock which is the wrong term here)

For clarification: The protection regions will always be created when a dungeon spawns. The config option "protectionSystemEnabled" only affects wether it will actually check if a player is allowed to place, break, ... blocks inside a region. The region always gets created, reason is that it is not just used for the protection system. Also the region files don't get converted to the new versions sadly.

Overall a lot of the numbers you claim here seem fishy to me, so i think it is appropriate to request a Java Profiler report so we can actually see what is inside the memory.

OK, seems like the code for saving the protected regions was not updated in a while. In fact it deleted every single protected region file to then save all the regions, now that i saw it the long saving times seem possible.

I did something in this commit that should solve that problem. I changed it so it only really saves the zones that got changed and does not delete all files to just create them again. Also that happens async now, so it should not block the main thread. However, this is not tested and will be published once i tested it.

So I did some testing:

1. I installed cqr, chunk pregenerator and several performance mods.
2. I set the dungeonSeparation from 20 to 10 and I set the chance of every dungeon config to 100.
3. I created a new world.
4. I ran the command /pregen gen startmassradius square 0 0 10000 and waited until all chunks and all cqr structured were generated
5. I pressed escape several times to save the world. It consistently took 200ms to delete all the old protection files and 3500ms to save all protected regions. Also there was pretty much no increase in ram usage while saving the world multiple times.
   So that means yes the saving of protected regions can definitely be improved. But my results do not match what was reported.

I will try generating a world with an older version of cqr and then I will update cqr and load the world.
Edit: Apparently I only generated 200x200 chunks and not 20000x20000. That means the reported waiting time sound possible.

A few tasks/questions for you (some things might already be requested):

1. You checked the I reproduced the bug without any other mod's except forge, cqr and it's dependencies checkbox but in the To Reproduce section your first point is This error is produced on the Tekxit 3.14 (PI) modpack by Slayer5934. Please make sure the error is actually reproducable with only forge, cqr and llibrary installed when creating a new world.
2. You said the console is full of cqr errors. Please provide the log containing these errors.
3. You said the server crashes. Please provide the crash report.

A few side notes:

1. It is intended that protected regions are generated despite setting protectionSystemEnabled to false.
2. The checkAndUpdateDeprecatedStructureFiles config option refers to the structure files found in minecraft/config/CQR/structures. When enabled all nbt files inside that folder are checked if they were created with an old version of CQR and if yes they are updated and the file gets overridden.
3. When a protected region is created from a file which was created with an old version of cqr then it posts a warning into the log. When the protected region is saved again it uses the new file format and thus when the files are read again it shouldn't post any warnings.
4. The structure files caching only increases the ram usage by about 50 MB with the default config. So this should really change anything.

Id have to dig through the log files and find the ones where it was spamming the version errors. But you already know what those version errors are, so giving them to you wouldnt solve anything.

Basically when you use an OLD version of CQR, pregen a world of 15k or more, and you STAY on that OLD version. The server runs smooth, no issues saving, no huge memory leaks. There is a slight increase in memory by about 500MB difference from a fresh world, but thats it.

Now, take the same world. And put a NEW version of CQR on it. All hell breaks loose. Right after boot up your now using 500-600% ram then what the previous version was using. And the console is now being completely obliterated by version errors whenever its trying to read the CQR files. It will print these errors endlessly, even if the protected_regions files are deleted. The only solution to ending these errors is deleting the structures.dat file in the world/data/cqr.

By doing this, the errors will stop. But the problem remains. The ram increase is still there, and now, if you save the world, it will take up to 60 seconds to save it. During that 60 seconds, the SERVER or CLIENT does not respond at all, everything is locked up. To a server this is now a "DEADLOCK" as its now reporting locked threads. Then it as it 'saves' the world the ram will sky rocket. Every save is an easy 3-5GB of ram. Until it reaches the max ram usage set to the server or client and then causes a JAVA HEAP CRASH.

This ONLY OCCURS when you change versions of the mod. Pre generating the world, saving, loading, booting, etc on a world generated on the same version of CQR works fine, no problems. Its only when the mod is updated on that world is when everything gets set on fire.

As for checking i ran it with only cqr, that was a mistake, i dont know why that was checked off.
I hope this clears up some things @Meldexun .

This is being ran on a hybrid server called CatServer, and im also using Adopt Open J9 (Java 8 + OpenJ9 JVM) for my java.

Since you are really getting rude now (and i guess you assume that we pretty much have the intelligence level of bread as one can read through your phrasing of sentences), i need to ask you to please remove the mod from your servers.

First: We do not support CatServer, Magam, etc (all those hybrid servers), so we won't continue to work on this.
For the version conflict things: The ones caused by the region files may be fixed, but there's no guarantee.

A deadlock normally is when two threads each wait for the other. In your case the thread is just blocked, being busy with a single task.

Since we do not support old versions or hybrid servers and don't see it as a big problem to delete the old world we won't work on this.

Only thing we will change is the version conflict stuff, so it only occurs once, meaning on load when it will convert the files (will be added for all other files sometime in the future).

Dude i am not being rude, i haven't insulted anyone in this entire thread. You have done 100% of the insulting here.
I have replied on my setup, the problem, how to reproduce, the effects of the problem, and the errors given to me.

I seriously do not know what your problem is with me, i have done nothing to you. I dont know you, and you dont know me.
Yet from the start you have taken this report as an attack on you personally. I never referenced you once. I never insulted you, your team, or your project once. I have simply reported a problem i am having.

Thats it.
Also why did you edit my reply saying your response was un-professional?

Hm that is certainly true, i guess i overreacted on this. Problem is that to me some of your phrasing sounds offending, i'll however try to not get offended from now on.

Anway, back to my response:
What we'll fix:

• version errors of region files
• region files actually getting converted

What we won't fix:

• world conversion from old version, however, i guess this is related to the version of the region files so fixing that will also fix this.

What offended me is this: It appears to me that you are talking from a high horse to us (i don't really know if this is the right translation though) which i don't really like.
One question: When updating the mod, did you follow the update instructions and deleted the cqrepoured.cfg file and the CQR folder inside the config folder? If not, then please read those instructions in the future, they are located in the changelog.

Im not talking highly, im explaining things to the best of my ability. I have reported issues for over 10 years. I do not know java, but have java developers for my network. Regardless if i find an issue on my servers i report it to the team that i believe where the error is originating from.

There has been times ive reported a problem to one party, and it ended up being a conflict or issue with another, in that case id go file a report to the other party as instructed. You insulted me when you said i didnt know how to read a log. Didnt know what a deadlock was. Claimed i was being rude off the bat when i never said anything insulting or disrespectful in any way.

Text cannot show emotion, it cannot give anyone an idea of how someone is saying something. Words can. I would justify your treatment of me if i came on here accusing you of sabotaging my servers, insulting the project and yourself, and using vulgar language. But i have done none of that. And i never have in my hundreds of posts on github regarding issues.

I have typed this way to many parties and many people, including teams like AdoptOpenJ9 themselves to figure out a problem and a solution. I have not had anyone treat me the way you have here, ever. Even after i apologized and told you i had no intention of offending you, you still attack me.

Back on topic: Yes i did wipe all files as i stated multiple times above it was the only way to update to make the errors stop. The configs and data folders were wiped.

As for not supporting CatServer or Magma, i stated i was able to reproduce this on my client by copy/pasting the world files into my client. So the server software not being supported is fine with me, as im able to reproduce it without the server software.

If youd like i can have one of my staff members who assists me with my own modpacks to take over this discussion and testing.

OK, then it really was my end being unable to read your texts properly and i want to apologize for that, problem is that i tend to overreact on things when i (falsely) think someone insults me. I've read over your texts again and yes, you're correct, you did not insult me at any point. I really feel sorry for this and i will try to be more factual in the future. I respect you for still being factual towards me and the rest even though i really was rude towards you.

Back to topic, ok it seems like i overread that you tested it on the client alone. Thing is, i'm pretty clueless what causes the huge ram increase, i will try it myself today.
Just to verify, those were the steps?

1. Pre-generate huge world with old (i guess the last alpha version of cqr?) cqr version
2. Stop game, exchange old cqr with version 2.1.3
3. Fulfill update instructions (meaning deleting CQR folder and cqrepoured.cfg file inside the config folder)
4. Boot up game
5. See ram increase during the start
6. Enter world
7. Let the world save
8. See huge ram-need increase

Oh, could you alse send us the exact version number of the old cqr version?

The original version was: 2.1.0B
The new version was: 2.1.3B

And yes those steps would be correct. You would be able to see a large ram difference from a fresh world on the new version compared to the old world on the new version. The bigger the world save, the longer the save time. The bigger the world, the more ram spikes during a save. But this does not occur if the world is generated or saved on the same version. This is simply an update issue.

ok, thank you. I'll try to reproduce it later today

OK, i reproduced the error with a smaller world, it is not that significantly visible on the size i tested on but it is visible when using a profiler.

The problem seems to be bound to the following:

• Whenever it saves the world, the mod deletes all files in the protected region folder, then it saves all protected regions again, this causes CPU load and is slow since it needs to wait for the harddisk to complete the tasks, this seems to be the cause for the main thread to get blocked
• Since the region files don't seem to get converted currently, it prints a lot to the console, which like above causes the main thread to wait until the mod has printed to console which can take long with many files
• Both points above scale with the amount of dungeons generated so it gets worse on larger worlds

We made a few changes to the main branch which do the following:

• It only replaces the files of regions that have been modified, so it does not delete all files
• That whole process happens on a separate thread, so it does not block the main thread
• If i saw it correctly, then it does convert the region files now, so less console spam

I suggest the following: We will release a patch in a few days, it would be cool if you could try to reproduce this issue with the patch version once it is out. I tried it with a patch build a few minutes ago and the issue seems to be gone. However, it could still cause a little bit of lag on the first save when a lot of dungeons were generated cause it then needs to first save all the files, but since it happens async it shouldn't have a large effect on the main thread.

I think it's just too many protected regions that cause the OutOfMemoryException. Can you check how many files are inside the minecraft/saves/world/DIM-1/data/CQR/protected_regions folder?

Zero, there isnt even a CQR folder in an y other dimension other then the main overworld.

I still have some points to note:

1. Id have to dig through the log files and find the ones where it was spamming the version errors. But you already know what those version errors are, so giving them to you wouldnt solve
   No, it is still necessary. CQR can post a lot of different warnings. I can only assume what warnings you are talking about. Also I don't want to see the log to just read this one warning which gets printed. With a log one can check what happened at which point and what warnings it created.
2. How many files are inside the minecraft/saves/world/data/CQR/protected_regions folder?

Before i deleted them all? Around 8700.
And really, the only error i was getting was warnings about the versions.
Thats and a bunch of stuff yelling about missing things for other dimensions other then the overworld. But im guessing thats part of the "preventing CQR from generating on other dimensions". Still would be nice to silence all that debug though.

I tested it by:

1. Creating a world with cqr v1.21.9
2. Generated a few dungeons
3. loaded the world with cqr v2.1.3

For me as expected it prints a warning when trying to read an old protected region file. That happens one time because after reading the files they get overridden when the world is saved again. From that point it does not anymore print warnings when trying to read old protected region files.

That means for me the warnings work as intended and there is nothing to do. When you think it somehow prints the warnings too often then the log is needed to check what is happening.

Uhm what exactly are the "missing things" erros?
That does not seem to be part of "preventing CQR from generating on other dimensions", except you use some custom mod for that. In general CQR waits for a chunk to generate, then checks if that chunk is on a certain grid. After that it searches for spawnable dungeons in that specific chunks by checking biome, dimension, coordinate, etc. So i think it would be interesting to have the whole log file. If you want a dungeon to only generate in certain dimensions: It has a list of dimension ids in each dungeon config that defines in which dimensions a dungeon can spawn.

Im sifting through logs now, trying to find this stuff.

Can we have your configurations of CQR? Something seems to be wrong there....

The warnings about loottables missing are normal, it can be ignored.

Im using the configs in the pack.
https://www.technicpack.net/modpack/tekxit-314-official-1122.1718228

I have edited the CQR config to disable protections and caching though.

only the main config or the individual dungeon configs too?

Main config.
The one in config/cqrepoured.cfg

ok, thank you

Finally found the log with the errors:
2020-12-20-13.log

OK, the "Config string is invalid! Not enough arguments!" comes from some loot configs in the pack being invalid.
I'll add more information to that debug line so in the next patch we can see what exact line is causing problems.

I suggest to leave this now as it is and we wait until you tried the patch version once we release that.

You can see in that log how it used up all of the servers ram (was set to 16GB at the time)

You can see in that log how it used up all of the servers ram (was set to 16GB at the time)

Ye, isee. I guess it really is bound to all of that deleting and creating files from the protected region files leads to this. It does that for all the files, doing that it blocks the main thread and the memory runs full cause other things can't do their stuff. And with ~9000 region files that can be a lot and it will take a while for it to finish....

Alright sounds good, thanks for looking into this for me.

I see a lot of commits related to this, any progress on a new test build to try out?

Not everything is related to this.

In general builds are published on tuesdays and only when we feel like it. I suggest you to be patient and wait

I am being patient. I was just asking if there was any new builds i could test thats all.
Man not everything that's posted is an attack on you, chill.

I don't think it is too much to expect users being able to check the mod's website or it's curseforge page for updates themselves

We published a new release today, please test if the issue still exists.

This... Might take all night...

We did say it might take a while, since the world is kinda huge, there are a lot of files to update.

Anyway, you can either wait until it has updated everything or recreate the entire world, which might take longer.

Well the point of the test is to see if theres saving problems or memory leaks AFTER updating to a newer version.
So i guess ill just wait it out.

OH! The percentage is wrong.
It keeps saying 0.XXXX% but the numbers after 0. is the actual percentage.
It just finished.

So the last % on that screenshot showing 0.19% is 19%.

Server seems stable at 2% cpu use and 1.1GB of ram use after 5 save-all's.
Saving is almost instant.
Major improvement from the last version.

Ill leave the server running for the rest of the day to see if anything changes over time. Then do a test with a few staff online running around.

Everything looks good, thanks for your help resolving this problem!
Sorry we had such a rocky start.
Gonna close this now. :)