[MOD = 3.6][MC = 1.11.2][Questions about pre-generation]
Endmateria opened this issue ยท 22 comments
Sorry, I know you have written extensive details on the usage of this mod, but there are still a couple things I'm curious about regarding the mechanics of this mod.
- Does anything undesirable happen if you use the pre-generator on the same chunks multiple times? Like does it register that it's already added buildings in that area, or does it ignore that fact and try to place more?
- I'm getting a moderate amount of cascading generation lag (in pre-generation). Does your pre-generator normally cause that, or is that more likely caused by mods I have installed that may have poor (runaway) generation code? If the latter is true then do you know which types of terrain generation assets are the usual culprits? Mods that add large batches of assets that overlap? I mean, I've got the Coral reef mod that spawns in glowing coral in patches, Quark, which manipulates the placement of underground stone and adds huge underground caves/biomes. (I'm willing to bet it's one of those two). Better Bedrock Gen, which makes all the bedrock flat. BOP which does a lot in and of itself. And then the rest are just mods that add small amounts of ore here and there (probably not the cause). Or could it be that I simply have too much crap generating? hahahaha
- If you start the pre-generator and select a size without selecting a starting point, where does it start?
- Of course it does not. It generates one time per one chunk population. All structures that started in specific chunk will be spawned during chunk generation. Each chunk generation always happens one time. So you can pre-generate multiple times, but any chunk in such range which has already been generated it just loads into memory, not generate it again.
- Go to server.properties and set max-tick-time=-1. It happens because of structpro because pasting large structures cause processing of falling gravel, sand, water, lava, mobs etc. But I'm not totally sure that everything is absolutely correct.
- It starts in (0, 0) chunk
Yeah, I read just about everything in your information section. Got the max tickrate set at -1 already... that's why I asked if it was being caused by other mods. Might just be the amount of mods I've got generating stuff... https://www.reddit.com/r/feedthebeast/comments/5x0twz/investigating_extreme_worldgen_lag/
I have to say that I additionally save and unload all redudant chunks each time I generate or load it to the memory to free RAM. So I believe the process can be optimized. For example I can skip already generated chunks. Wait for a future updates.
Very interesting article thanks!
Given how long the pre-generation process can take, having an option to "skip already generated chunks" would be great! And as for the other issue, I don't know, but I'm getting cascading generation, and there's a good chance that it's not your mod, but somebody else's that isn't optimised. I might have to slowly take off mods and play with it to see if I can find the culprit. Your mod is brilliant by the way, not being critical here at all... just want to get it working right while taking off the least amount of other mods as possible haha.
I checked out one of my previous 4097x4097 blocks (size =128 chunks radius) world pre-generation and what I realize:
- Cascading generation cause some rare third-party mods, because for my mod pack with ic2, biomes o plenty et all was ok and not cascading happened. To reproduce I need a list of your mods.
- Generation consume server tick thread to generate and force populate chunk, so it generates (X,Z), (X+1, Z), (X, Z+1), (X+1,Z+1) blocks immediately and no problems caused.
- Before determination that structure spawns in certain chunk starts surface roughness calibrator to detect roughness, liquid depth, biomes etc to say about spawning permissions. That cause also some chunks ahead to load, but it's not a lot. Big structures like deathstar spawn in 20 seconds that extremely fast. And if spawn conditions not permit, next time it will recalculate much faster because of all in RAM.
- After some time after pre-generation starts (for me it started on ~63000/66000 chunks progress) Massive lags appears and It's because statistically one of the structures with lots of sand and gravel spawned in the sky and start falling down that slow-down server for a while.
Quark and Galacticraft, those are the two culprits I found thus far... I notified them. Farseek was also posting a lot of activity during the pregen... though I don't know why I even had that mod installed (it doesn't show up as a dependency anywhere). Let me run some more tests, I'm resorting mods at the moment. Oh well, here's a mods list in the meantime for shits and giggles... https://gist.github.com/Endmateria/19b081faa7ec6bfe82cd9c2203b111f4#file-crash-2017-05-06_16-24-58-server-txt
The list
java.lang exceptions happens when you trying to stop minecraft server and threads stops asynhronous (incorrect server stopping).
I was in game, the server forcefully stopped itself, or in so many words... crashed. It's done that multiple times. I'm of the opinion that your mod, Quark, and Galacticraft don't get along hahaha.
Here's radfast's response (from Galacticraft)... micdoodle8/Galacticraft#2689 (comment)
I read carefully what radfas says and I can say that structpro not cause runaway worldgen even when density is crazy overcap, because of next features:
Structpro generates all structures on population stage that start in (x,z) chunk that growth only in positive axis directions (+x, +z), so you can see how appears rectangle (x,z) - (x + width, z + length) in the world. And such process not begins to populate border chunks. That means that newly generated chunks fires only direct neighbours to generate (not populate) and the process stop there. To continue the process players move forward and newly generated chunk that is not already populated populates itself and process continues.
The general question here is why (8, 8) +-20 blocks spawning can't be realized.
It can not be done because structures that have width or length more than 20 blocks need information about biome and soil or liquid heights in a specific region, so it can not be pre-calculated until all chunks not generated yet.
You can't get biome, a real height map of water soil and other block types in a specific region in the world until you generate it - you can't prevent some additional chunks force to be generated, which number is how much chunks contains one schematic and it's not a lot.
Since we determined that no special runaway process starts I close the issue.
Well, I took off some of the mods that cause cascading world gen... Coral reef and Tinker's construct. Astral Sorcery has corrected the problem, just waiting for him to release the updated version. Your mod alone is still getting cascading world gen during pregen though. I'm running more tests this morning. Here's a short log excerpt of me pre-generating a 100 chunk radius (40000 chunks) on a new world map. In this short log, your mod has 18 hits worth of cascading world gen lag. Could it be the structures you chose, and not necessarily the mod itself? https://gist.github.com/Endmateria/99b4a59ebcdb10e1c0abcb2195142f4b#file-fml-server-latest-log
I saw your gist and that's totally not related to structpro.
Cascading world gen not happens except rectangle zone where schematics paste.
Yeah, this is what I got... similar. https://ibb.co/mwFtMQ (40,000 chunks pre-generated).
This small squares is a space that schematics required to spawn or to check spawn conditions. Everything is correct. Nothing critical, that's the way how the mod works. There is no other way to implement or improove logic, because mod have to know terrain roughness, height map, liquid depth and biome to say spawn or it's too rough area or too shallow etc.
I gotcha, makes sense. The cascading worldgen was mainly from other mods, whom I've opened up issues with. My major issue was with the server crashing ingame.... but that only seems to happen when you go near the edges of the world that have not been pre-generated. Have you considered adding a world borders option to prevent the server from crashing?
Hey man, saw the options that you added to the /structpro help menu. Very nice. And thanks for troubleshooting all of this with me. It's a very temperamental mod, but once you know all the rules, it works like a charm. I would recommend specifically warning people on your webpage that the server will crash if you don't pre-gen properly (since crashing seems to be everyone's primary complaint in your issue tracker), and I also recommend telling people to put a world border around the pre-generated area. Hopefully that will help you avoid having to address all of these concerns for someone else. Anyways, thanks again... great mod!
Ok, so the last two times I've run the pre-generator with large sizes (1000+ chunks), I've gotten an "out of memory error" that crashes the server. It ran for about an hour before crashing. I've already given the server 8 gigs of ram. Do I give it more, or is there a memory leak somewhere? Or is it because I didn't set the "step" logic to 16, which might make the generator accumulate a garbage heap quicker than it can dump it? I'll set the step at 16, increase the memory to 12GB and try again...
https://gist.github.com/Endmateria/02e99417a3816172e34ab9c89490c021#file-crash-2017-05-09_01-29-27-server-txt
Yeah, even at step=16 the generator still gets slower the longer it runs... then eventually it crashes with a "TickNextTick list out of synch" error...
Crash report: https://gist.github.com/Endmateria/442944648b49b3264c369b668ca78228#file-crash-2017-05-09_15-53-18-server-txt
Log: https://gist.github.com/Endmateria/9598ff5312b2ff2f1a7c4a56043b72a6#file-2017-05-09-7-log
Also I get this error occasionally while generating using the "progress" feature. If it's just getting this error because the generator is passing by a structure that it already placed that is no longer loaded into memory, then that makes sense and I'll disregard. (only happens after using progress/resume). https://gist.github.com/Endmateria/b5aa25206a4635db045cc2b861ef15b0#file-gistfile1-txt
I think the "java.util.ConcurrentModificationException" is someone else's mods conflicting, cuz I was still getting that error after removing your mod and starting a fresh world... so I would ignore that one. Also, I guess the memory issue with pregenerators is a common problem... not singular to your mod. Maybe if the step size was smaller, it wouldn't crash so much, I don't know, I'm not an expert on the subject.