Sorry for the long post.

Several months ago, I was working on a fairly large modpack based almost entirely around adding new modded blocks and biomes to Biome Bundle. So I don't waste your time, I'll just say that one big feature was a dimension filled entirely with caves. This was handled using giant BO3 structures, which was problematic both in the way of performance and also in getting other structures to spawn correctly. The project was eventually abandoned due both to my own stupidity and another project I was working on, which was originally designed to be a small mod that added different stone variants for each ore type I was using at the time so that the caves would look nicer. (Un?)fortunately, that project grew way out of hand and has shifted my free time priorities >> 1000.

I'm revisiting this whole caves idea, but this time, I'm looking for something better. Over the past week, I have begun to dissect and understand how Mojang's cave generator produces its shapes, as seen underground in every Minecraft world since long before launch. You can view my progress on GitHub, as well get more information and a compiled build on CurseForge.

Basically, I would more than happy to implement as many of these features as I can into OTG (after making a few more improvements first). I just need your feedback on a few things first:

• Whether you feel these changes are practical and fit within the mod's overall meta.
• Whether you would prefer to wait for 1.13.
• Whether you would prefer an entirely new preset format similar to *.BO3 or *.BC instead of *.json

If so, and if there are no issues with the above questions, then:

• Whether you would prefer these changes to remain biome-specific, as I have them.
• Whether you're comfortable with some currently very questionable, beta features, such as the BlockFillers / decorators.
  • These are used to randomly layer different blocks on top of or inside of cave walls in any direction, which sometimes requires storing the info in new files for later use, i.e. when the given chunk has not yet generated.
  • This theoretically decreases world generation speed as more chunks get added to the queue. I can say that, having explored at least 20k blocks in one direction, the impact is not noticeable; however, it definitely still needs work.
• Any other practical issues and whether you feel they could be worked around.

As this will take me so long (at least one week after I finish making adjustments to the original mod, fingers crossed), I will not submit any pull requests or even begin working until I hear any concerns you have.

For now, here's a quick rundown of what this will entail:

• Biome-specific cave presets with the following options and more:
  • Tunnels
    • Slope control
      • Starting value, randomness factor
    • twist control (the amount a slope changes per iteration)
      • Linear and quadratic scaling, starting value, randomness factor
    • Scale control
      • Horizontal and vertical; linear and quadratic scaling (vertical needs work), starting value, randomness factor
    • Noise control
  • An entirely new cavern generator
    • Can be used alternatively or in addition to the tunnels
    • Octaves, frequency, height control, etc.
      • Height control supports smoothing. Currently uses 2D noise, but I also have a slightly more efficient method which works by reducing the scale near height limits.
    • Almost no additional overhead when used together or instead of the tunnel generator
      • Between 0.5 and 2.0 additional seconds world gen time from vanilla, r = 12
    • See this comparison of my original caves preset (mind the light--sorry) to the new caves generated procedurally by the mod.
  • BlockFillers
    • A much more efficient way to cover caves in grass, flood them with water, add stalagmites and stalactites, place vines on walls, etc.
      • This is because fewer blocks are tested for placement
      • Almost zero noticeable performance overhead.
    • Supports calling on other generators (presets) to spawn branches
    • Could easily be adapted to allow spawning BO3 and other structures, neatly and guaranteed to be within cave boundaries.
  • A command for reloading presets, much like "/reloadBO3s"

I hope this wasn't too much to read. I was afraid my code was just garbage, but I think I've made a sizable volume of progress at this point and I can't wait to hear back from you.

Edit (this may be relevant): Cave Generator actually does not break seeds by default. I'm not yet sure how well I can do the same with OTG, though.

Hey @PersonTheCat, I see you're still hard at work, how's it going? I'm finally getting back into OTG after some RL downtime, finishing a big release atm which should be a be major milestone. Also finally cleaning up and refactoring things, so it's good to be back. How's it going with CaveGenerator and what are your plans for the future?

Hey, thanks for checking in. It's going fairly well. I'm currently in the middle of a full rewrite of CaveGenerator. This includes optimizations, some new features, bug fixes, an improved preset format using a custom version of hjson-java, preset checking to notify users of bad configurations, integration of FastNoise to allow users to control noise types when generating caves and various decorations, better error handling, a focus on null-safety, and some extremely hefty improvements to code structure and readability.

Unfortunately, I never intended to do anything quite as time-consuming as a full rewrite. My plan after opening this issue was to finish a few more features and then rewrite the mod as I integrated it into OTG. However, CaveGenerator started to receive a little bit more attention than I expected. While it still isn't that much, it was enough to prompt me to get a little more serious about it. So, that's where I am right now.

At the moment, I'm about 80% done with the rewrite. I just have to rework a few more features and then fix two or three major bugs:

• WallDecorators not getting full coverage between chunk boundaries. I've gone through several different solutions, but it's taken me until now to almost find one that doesn't have any drawbacks. I'm almost done.
• Giant caverns carving holes on ocean floors. This is especially frustrating because my code isn't actually responsible. This problem occurs when the ocean floors are decorated with gravel on a later event, replacing the topmost layer and either leaving holes or unstable surfaces. Sadly, fixing this requires me to work around the problem more so than avoid it altogether. It sounds like an easy problem to fix, but I've been repeatedly surprised, so far.
• Caverns leaving flat walls when being cut off between biomes. I still have code for smoothing this out, but the problem is that one chunk of smoothing is apparently not enough distance to really do a good job. I worry that anything more will have too much of an impact on performance.

While 80% is pretty close, I also recently started a new job, so things are moving slowly. Sorry for taking so long to get through with this. The base code is still fundamentally very similar to how it was when I first opened this issue, but the process of polishing that code has unfortunately consumed a lot more time than I ever expected. :/

Haha, sounds a lot like OTG, and many other mod projects I'm guessing, welcome to the club ;). No worries, look forward to seeing CaveGenerator when it's done. I'm guessing your problem with WallDecorators is because of "connecting" blocks like torches/vines/fences/walls/redstone etc, that look for a neighbouring block to stick to when placed, don't connect to blocks in unspawned chunks? That's a problem in OTG BO3's too, eventually I decided to spawn everything "in place", so not updating the block after its spawned and assuming that the block is spawned "already in the correctly updated state", which in the case of OTG BO3's should be true. In any case, good luck polishing, let me know if you have any updates! :)

Thanks for the advice! The WallDecorator problem is actually related to replacing blocks in cave walls. In order for me to do that, I have to look into neighbor chunks in order to determine whether a block is actually a wall block. However, I can't do that when working with ChunkPrimers, because those chunks don't exist yet. The solution I'm currently working on works by testing to see whether the given coordinates would exist inside of the current cave section. That does let me place blocks on walls, but not in walls, because placing blocks in walls requires me to come from the other direction.
That is possible, though, and I'm almost there, so I'll definitely let you know!

Maximum cavern height is now generated using a height map that gets added to the existing terrain height. This produces some interesting shapes and, more importantly, fixes the long-time bug of caverns carving holes in ocean floors.

Just finished updating Cave Generator to version 0.13. A download and its changelog can be found on CurseForge. At this point, my plan is to wait just a bit for some feedback about how smoothly everything runs for everyone. In the near future, I should be able to clone the master branch for OTG and start working on changes. I would plan to start with just the custom tunnel / ravine variables and a noise generator used for generating underground caverns. I would really like your feedback beforehand, though.

• It should be possible to have multiple tunnel generators enabled simultaneously in OTG, just as it is in CG. My only problem is that this will take up an enormous amount of space in the current biome configs. I can add tunnel settings in the world config, but overriding the global config per-biome also needs to be possible, simply because it is possible in-code and it produces very interesting results in-game. What I need are your opinions on how that should be done. You may want to take a look at how tunnels are managed in the vanilla preset on GitHub to get an idea of how many parameters there are. The fields you're looking for are listed inside of [root].tunnels
• The cavern generator now uses FastNoise to handle noise generation. This has a couple of its own problems.
  • The entire FastNoise library (slightly modified) is included directly inside of the jar file.
  • The use of FastNoise is inconsistent with how noise is generated elsewhere in OTG, and thus you may want to consider more thorough changes involving it.
• FastNoise is highly beneficial, however, simply because it allows the user to choose from a variety of configurations. This means that users can choose between simplex noise, perlin noise, and use a whole bunch of other setups to produce their own shapes. None of this can be achieved without FastNoise.

I'm sure there are several hour long conversations that can be had about altering the cave generation algorithms for OTG, but I figured these two subjects would be a good start for now, since I can absolutely handle their integration with no issues whatsoever.

Love the pics btw! One more note btw, it's a basic rule for OTG that we don't add any assets to the game (blocks, materials, entities etc), I hope that won't be a problem.

Hey @PersonTheCat, MCPitman has been asking me for better caves for ages! I simply don't have the time so would much appreciate your help. If you can implement caves in such a way that it is optional, doesn't break existing features and the code is (reasonably) self-contained so it can be merged without much trouble then I have no objections at all. I'm wondering though, will these caves be procedurally generated from pre-fabricated (and even customisable) parts? Then yes I imagine you'd need something like BO3's. If it's more like the normal cave generator which uses a relatively simple algorithm with configurable variables then I don't think you would need anything like that, you could add the settings to the biome configs, right? As for 1.13, at the moment I'm still hard at work on improvements for 1.12, I'll start worrying about 1.13 when Forge updates (there is someone who already said he's working on a 1.13 branch for spigot though). A note about cave worlds, there's a reason that the nether is only 128 blocks tall and that's performance, a 256 block tall cave world really slows things down. I still have to add a "128 height" option for OTG dimensions so users can make proper cave dimensions, After I'm done with the current improvements that'll be high on my priority list. I'm not sure about Block Fillers btw, it's always preferable to not have to save/maintain any state like unfinished chunks, but if it doesn't cost performance and doesn't cause problems I don't really have a problem with it. Ofcourse if the caves are optional and some of the performance-intensive features are optional then users can always balance their settings for the best compromise. Do you intend to test for Spigot as well btw or just Forge?

Cheers!

Okay, great. I'm glad to hear back from you. So, the cavern generator is actually extremely simple to implement because it's literally just a 3D noise generator. You can see the original algorithm here. I've just made a few minor changes and then added an equation to smooth the noise out a bit / fractalize it. The way I have it implemented in Cave Generator is literally just by iterating through x, z = 0; x, z < 16; y = 0; y < whatever height, where the existing stone just gets replaced if the noise is above the threshold. However, I expect that spawning caverns instead of stone (and not just replacing it) would be slightly faster, so I'll definitely look into that using OTG's chunk provider. This will most likely happen the the world config. I have a way to stop the caves from generating between biomes and it sometimes looks nice, but other times it just doesn't work, so adding it there will produce the easiest and cleanest results.

At this point, I also have a way for users to spawn stone in layers using 2D noise. This looks pretty good, but should ideally stay biome-dependent. In Cave Generator, this part happens whenever the cavern generator fails. I'll need to see if that's possible with OTG's setup, because it is the fastest solution I can think of for adding multiple new types of noise to the world.

The block fillers were actually super fast and didn't really cause any issues for most users, but they had two major problems: 1, their size was limited; and 2, multiple blocks could get written to the same BlockPos, which was just inefficient. I've fixed all of this by imitating the way ChunkPrimer stores its blocks. I've also written a custom read/writeObject() method so that only the indexes which contain meaningful information are ever (de)serialized. However, while it wasn't notably slower before, it definitely is now, so I probably will not bother you or other OTG users with it. On the other hand, placing blocks on ceilings and floors (as well as filling / flooding caves with water, gravel, etc) using a similar method is still much, much faster than using BO3s, so I will include that for you to at least consider. I also plan to write a dedicated stalactite generator for placing multi-block stalactites and similar decorations more easily, but I haven't even started on that, so I can't quite consider it for OTG just yet.

The rest of the mod is just Mojang's normal MapGenCaves, except I've identified the impact of most variables and even have screenshots and GIFs to demonstrate their effects. Users can change them and even have a few new options that didn't previously exist. I will need to completely rewrite CavesGen to support these changes, but I'll do my best to avoid breaking existing functionalities and even test Spigot for you.

Give me at least a week before I can even get started. Another group of developers has contacted me for help with overhauling cave generation for their mod as well, which I plan to start on today. Once I'm done with that, I may start with just the cavern generator, which will be super easy to implement and then start on the rest shortly following that.

Edit: and of course, there is no need for new blocks to be added! That first album was just an old preset I was using.

Actually I meant just the standard boring OTG cave generator, I did profiling for CPU hotspots using visualvm and found that the caves generator was creating just as much load as basic terrain generation. Most likely because basic terrain generation doesn't need to manipulate existing blocks or check for the existence of neighbouring blocks etc. VisualVM is really easy to use btw, just download it from https://visualvm.github.io/, start it, start a MC client, you'll see the process appear in the process list for visualvm, rightclick it and pick "sample" (not "profile"), then try out the CPU and memory tabs. VisualVM detects CPU hotspots (code that is used very often and/or causes a lot of load) and shows you how many and what types of objects are in memory. Be aware btw that attaching the cpu profiler/sampler can sometimes affect the process and cause a crash, that's not necessarily a problem with your code.

Hey @PersonTheCat, sorry for the late reply, I see you're still active, how's it going? I think it'd be great to make the mods compatible, but not necessarily integrate the two projects. Since you're developing Cave Generator as a separate mod now, I'll close this issue (cleaning up the git atm). My biggest concern with a cave generator btw is performance. I've been doing a lot of profiling lately, and even the current, basic, cave generator is causing significant load during generation, as much as the rest of the base terrain combined at times. As for noise, OTG uses a few different noise generators, much of the code is lifted from MC. Unfortunately we can't just change noise generation for biomes, as that would break everyone's seeds. But there's no problem with using different noise generators for different things. I may take a look at FastNoise in the future, still have a lot more things to clean up and fix before then though :o.

Just out of curiosity, which presets did you try using? From my own experience, it tends to be the WallDecorator feature that causes the biggest hit to performance. It's horribly optimized. Seeing as OTG can already handle this sort of thing in better ways, this would not get ported over. Just the ability to change tunnel shapes, angles, etc. as well as optional, noise-based cavern generation.