Describe your suggestion

buildIgnoreBlocks and buildIgnoreExisting do the opposite of what I am suggesting. They ignore blocks in the world that do not match what is in the schematic.

I want the opposite, I want to pre-process the blocks in the schematic, interpret the blocks in the schematic. I want to add a new concept, that a particular block in the schematic tells baritone not to change anything in the world. I do not want that to be a fixed block like Air. I want it to be a settable block. When this block is found in the schematic, baritone should take no action, baritone should say, I do not care what is in the world, it is OK as it is, do not try to change the world at that point. I call this the I-do-not-care variable. Usage would be:

#I-do-not-care [{block_ID}/clear]

Usage to see what the I-do-not-care block is set to.
#I-do-not-care

Usage to set the I-do-not-care block to be sand
#I-do-not-care sand

Usage to clear the previous definition of the I-do-not-care block so Baritone returns to interpreting schematics literally.
#I-do-not-care clear

When a designer intends a schematic to use an I-do-not-care block they must choose the block at design time, and communicate this choice to users of the schematic. The author of the schematic might save the name of the schematic to signal to the user what to set the I-do-not-care block should be:

My_favorite_schematic_IDC_Sand.schematic

There are so many blocks available in Minecraft that it is possible to choose any buildable block that does not appear in the intended build of the schematic. So say I want sand to be my I-do-not-care block. I can save the schematic, and use the schematic without the I-do-not-care-block set, and I see the oak planks being built -- this allows me to see what is in schematic. Then I set

#I-do-not-care sand

and I build the schematic again -- this time not only does it not build the sand, it does not do anything at the locations that there were sand in the schematic.

At run time, the user would know they are using a schematic with an IDC block, and they would have to set the I-do-not-care block variable appropriately before running the schematic.

The use I would put IDC blocks to is, when I want to build a schematic box to enclose a large structure, where part of the structure contains Air, I can fill the portion of the box that I do not really care about with IDC blocks. It means I can create schematics that are effectively not right-prisms. It means I can build triangles or pyramid-shaped schematics, and that part of the shape that is not used in the bounding-prism box is just filled with IDC blocks. This would allow me to make effectively non-prismatic schematics with any current design tool for creating schematics.

For instance, but certainly not limited to this use, when I am building highways in 2b2t, on axis-aligned highways, I make my schematics 6 blocks long in the highway direction. An axis-aligned highway is a right-cylinder. I only need to define its shape in a right-cross-section, which for minecraft, is a one-block-wide slice across the highway. Unfortunately, one experiences glitches where blocks are laid and then disappear. Baritone's behavior is to re-place blocks in the current schematic image that are no longer present, however, if one uses #buildrepeat, when the Baritone advances by the increment, when blocks from the last lay of the schematic, that are not in this lay of the schematic disappear, they stay as a defect after the passage of Baritone. HighwayTools (the KAMI/Blue module) uses 2-block wide schematics, and under some conditions will leave glitches behind. With a 6-block long schematic, every block is in 5 consecutive advances of the schematic... In practice, only the blocks on the leading edge of the schematic are laid in a given cycle. If there are no glitches, there is no need to re-place any blocks. As glitches appear later and later, they may be replaced in one of the 5 previous lays. This is a solved problem -- glitch-free axis-aligned highways -- just make the schematics longer than they have to be.

This does not solve the problem for any highway at any angle where one does not want to determine what appears in certain blocks in the schematic -- if one does not have the use of IDC blocks. Without IDC blocks, there is no way to have an entirely overlapping schematic that advances with buildrepeat 1,0,1 for instance without including a lot of other blocks that either Baritone must place sometimes or must remove sometimes. When the schematic is limited only to blocks that must be determined, the blocks in the wing columns of the schematic only appear in one lay of the schematic as it advances along the other diagonal. With the presence of glitches, there are a lot of blocks not proper in these wing columns after the pass of baritone. The problem can be solved by enlarging the bounding prism to contain multiple appearances of all blocks in every column but then the wings of the prism extend into spaces we really do not care what is there. No strict interpretation of the schematic will solve this problem.

I could set a schematic for a large underground installation, and set only wall skins blocks and the contained Air blocks to be the desired block in the schematic... all the space between rooms and between tiers could be made IDC blocks, and then, if I base my schematic on a save from a world, and try to build it again at a different part of the world, Baritone is not trying to rebuild the random mineral deposits that might have appeared between rooms or between tiers in the original schematic. Anyone who has tried to make a schematic from an existing underground build has encountered this problem. One would be able to make standardized secret entrances (useful in Vanilla minecraft for sure) where all elements needed in some odd geometry are specified, but all unimportant parts of the bounding box are IDC blocks.

Freeing Schematics from the Strictly-Duplicated Right-Prism Prison is what IDC blocks are about.

As a further extension, I propose a second variable, As-long-as-it-is -- the full sentence would be I-do-not-care As-long-as-it-is (one of a list):

Usage:
#As-long-as-it-is [{list of block_IDs}/clear]

Usage to list the list of blocks that Baritone will apply IDC filtering to:
#As-long-as-it-is

Usage to set the list of blocks that Baritone will apply IDC filtering to:
#As-long-as-it-is Air Obsidian
If the IDC block maps to a place in the world that is Air or Obsidian, do nothing.

Usage to clear the list of blocks that Baritone will apply IDC filtering to:
#As-long-as-it-is clear

The behaviour would be, if

#I-do-not-care sand
#As-long-as-it-is Air Obsidian

If the place in the world that sand maps to is Air or Obsidian do nothing.
In the case that the location in the world is neither Air nor Obsidian the IDC algorithm does care and makes the block the first element of the #As-long-as-it-is list, in this case Air.

I would use this particular feature for, but not limited to, when I want to dig an area in preparation to laying Obsidian. If there is Obsidian in the schematic, and I do not have any, Baritone will fail. If there is no Obsidian in the schematic and Obsidian is encountered in the world, Baritone will spend a lot of time mining it.

With IDC variable, set to sand, I can just put sand in the schematic where I might eventually want to put Obsidian, and then say #As-long-as-it-is Air Obsidian -- Baritone would clear away any blocks which are not Obsidian. I could use the build schematic, but set the IDC variable to Obsidian and the #As-long-as-it-is Air Obsidian, and get the same effect.

Therefore:

#As-long-as-it-is Air Obsidian

is different from:

#As-long-as-it-is Obsidian Air

In the first case, the world gets more air for non-matching blocks, in the second more obsidian.

This cannot be done by buildIgnoreBlocks and buildIgnoreExisting because those interpret the schematic literally and then perform judgements on the world, and only with Air either in the schematic or in the world:

buildIgnoreBlocks
public final Settings.Setting<java.util.List<net.minecraft.block.Block>> buildIgnoreBlocks
A list of blocks to be treated as if they're air.
If a schematic asks for air at a certain position, and that position currently contains a block on this list, it will be treated as correct.

buildIgnoreExisting
public final Settings.Setting<java.lang.Boolean> buildIgnoreExisting
If this is true, the builder will treat all non-air blocks as correct. It will only place new blocks.

To do what I want to do, one has to interpret the schematic with a filter, and apply it to the world, regardless of what is in the world at that location, or only if the world does not contain certain blocks.

I think it would be a good idea to have baritone treat barrier blocks in schematics this way by default. This would simplify communication as no one would need to change their settings for most situations.

I think it would be a good idea to have baritone treat barrier blocks in schematics this way by default. This would simplify communication as no one would need to change their settings for most situations.

That is a good idea, and this way there is no need to add another mode ti it. Well of course some features mentioned by @HoratioGamer will be sacrificed, but will be much cleaner and simpler (and easier to implement).

My first objection to barrier blocks is, they do not behave like regular blocks when building them in creative. When one places them they are visible. Place several and break one, the others become invisible. Unlike the void block, at least they occupy the entire block volume.

My second objection to barrier blocks is they cannot be built in a survival world. There is no way to build the literal schematic (don't set the I-do-not-care block) in a survival world, tinker with it, test it, and then use Lunatrius and Schematica to re-save it as a schematic where one has the option to use an I-do-not-care block set. If what one is building is a machine, there are some things that do not work the same in creative as they do on some survival servers, even different survival servers might behave differently. By making it the barrier block, you remove the choice to use the I-do-not-care feature for a given schematic. Specifically, one is removing the option to build, test, tinker, test, save without having to take it back to creative to place all the flippin barrier blocks all over again.

OK if a command is too complicated:

#set I-do-not-care sand

Make it just a variable, not a command. I am not sure how much work it would be to add another command-line accessible variable to Baritone -- it seems the code already exists for other variables, just the name of the variable changes. If the code is written half-way sensibly, it is a copy and paste and change a variable name.

Then when it comes to the test:

foreach (schematic block in the schematic) {
  if {schematic block == minecraft:barrier}  continue;  /* skip to next schematic block */
  set target world block = schematic block
  try to place the target block in the world
}

becomes

foreach (schematic block in the schematic) {
  if {schematic block == i-do-not-care} continue;         /* skip to next schematic block */
  set target world block = schematic block
  try to place the target block in the world
}

With the as-long-as-it-is logic, it becomes:

foreach (schematic block in the schematic) {
   if {schematic block == i-do-not-care} {
     if {world block in as-long-as-it-is} continue;         /* skip to next schematic block */
     set target world block = [ lindex as-long-as-it-is 0 ];
  } else {
     set target world block = schematic block;
  }
  try to place the target block in the world
}

Sorry for the mix of C and TCL-ish pseudocode.

The added code to make it an actual variable does not seem onerous if one is going to open this can.

I-do-not-care --> schematic-styrofoam-packing-peanuts-block

I think this is generally a good idea, but I'd tweak the following few things:

• better names in the scheme of buildIgnoreBlocks and buildIgnoreExisting
• definitely make it a setting. A command is more complicated and not needed here.
• i-do-not-care should be a list like buildIgnoreBlocks
• omit as-long-as-it-is or make it a map of valid substitutes like the one planned for litematica
• don't try to parse filenames.

And two little notes at the end:
Your example of Baritone clearing a path from everything except obsidian wouldn't work because of #1925
Your first objection on barrier blocks is wrong: Barrier blocks are invisible, but as long as you are holding a barrier item, particles indicating their position are spawned (only visible to yourself). What you saw was the particles having different delay when appearing and disappearing, so the block you just placed was longer visible than the others.

After having a quick look at BuilderProcess I think i-do-not-care is trivial, as-long-as-it-is would be trivial and apart from a new command argument datatype a mapping of valid substitutes is trivial as well.
I'll look at implementing at least i-do-not-care tomorrow. Suggestions for a better name are welcome.

I am flexible on the name, so long as it is descriptive of what is is.

better names in the scheme of buildIgnoreBlocks and buildIgnoreExisting

I would have suggested better names for buildIgnoreBlocks and buildIgnoreExisting so one actually gets a clue they only deal with air blocks.... like:

buildIgnoreBlocks --> treat_these_world_blocks_like_air
buildIgnoreExisting --> only_replace_world_air_blocks

I-do-not-care --> schematic-non-buildable-placeholder-block
I-do-not-care --> schematic-skip-build-block
I-do-not-care --> schematic-unassigned-block
I-do-not-care --> schematic-absent-block
I-do-not-care --> schematic-dont-change-world-block
I-do-not-care --> schematic-null-block
I-do-not-care --> schematic-trim-block
I-do-not-care --> schematic-empty-block

And my favorite:

I-do-not-care --> schematic-ullage-block

look up ullage, I think you will like it, it is perfectly descriptive of the unused part of a container, the right-prism of the schematic being the container.

You get the idea, something that communicates that it is something in the schematic that changes the interpretation of the schematic, that it is an empty part of the schematic, or one that does not change the world.

(I was not intending any code to parse filenames, that was for the user to read and decide whether to use the recommended i-do-not-care block setting.... always intended that there be user choice, no code there)

And thank you ZacSharp for taking up this change.

Your're right, current naming of settings related to block-treatment while building is a mess (could be worse though).

I think I'll take your second suggestion and reorder it to buildSkipBlocks. I will insist on the "build" at the start, because I want it to show up with other build related settings in alphabetically ordered lists (pretty much anywhere).

Well I am just very happy to see this become part of Baritone. It does not matter what I have to type to make the magic work...

Finally the ability to make schematics that are not right prisms. It is like stepping into a bigger world. There are so many applications for this. Every time something did not quite fit in a box, now it does. One can make non-right-prisms like towers that lean to one side, or that simultaneously lean in both X and Z directions. One can make fancy stairs and ramps that cut through natural rock where the right-prism is long in the horizontal direction and skewing one end up or down in the Y direction to make it slope. Finally, building underground stables is not tedious. One can build things within tall circular and elliptical cylinder voids underground, even perfectly spherical shaped voids. One can recreate something like the great machine of Babylon 5 underground (using redstone of course) carved into native rock. One can recreate fantasy locations like the Mines of Moria, or real natural locations like the Mammoth Cave System, or Carlsbad Caverns, or any real interior void space that has had a 3D scan done of it. One can recreate the interior of one of the great cathedrals underground, or the crypts under Paris, or the London Sewers, or the burial chambers under the Pyramids. That is creative or survival.

In anarchy, where there are hacks to show everything and what is not natural will draw the attention of base hunters, one can make artificial caverns (overworld or nether) and artificial mineshafts that look indistinguishable from naturally generated ones -- because they were created from natural ones by simply snapping a schematic ! One just processes it through creative to get rid of all the mineral and other irrelevant features, and make all buildSkipBlock locations into the same block, not a hard step. That is the dodge of making the artificial (and convenient) look mundane. Really, the buildSkipBlock becomes a way to build air in any shape you want. One can also schematic familiar structures like desert temples or jungle temples, including the underground parts, and stick one anywhere. One can intentionally tweek them to have non-standard traps in them.

The very first use I will put this to is to create baritone schematics that can dig and pave nether highways at any angle we choose, and do so with minimal if any appearance of glitches.

I will just remind you of the hand-in-hand usefulness of the As-long-as-it-is concept:

If one interprets the schematic to take no action, provided certain things appear in the world, that is powerful. Besides all the uses I will put that to in highway building schematics, consider a simple mining schematic that is nothing but a solid block of buildSkipBlocks -- you say useless right? I say no.... imagine the ability to remove everything except the minerals from under a mountain. All you have to do is list all the minerals you want to preserve in the As-long-as-it-is list, with Air as the first element. Set up baritone to AFK clear out all the rock, leaving the minerals hanging in air. It mines out all the stone, granite, gravel, dirt, etc, everything not on the list, leaving all the selected minerals behind. It could be made not to mine silverfish blocks, in case later one wants them for some reason. One can then use a silk or fortune pick selectively on certain minerals, or go in and collect only the iron and gold for use in smelting to turn coal into experience orbs to selectively heal mending enchantment items when the iron or gold is removed from the furnace. Furnaces as bottles of enchantment just got easier.

I do hope you will consider implementing the As-long-as-it-is logic also.

nothing but a solid block of buildSkipBlocks

I'd use #sel fill air and #buildIgnoreBlocks diamon_ore,coal_ore,iron_ore,emerald_ore,lapis_ore,gold_ore,redstone_ore

And for the highway I'd build the pattern out of wood, select it, make sure there's no unintended wood in my selection, turn on build repeat and use #sel replace wood obsidian. Because ReplaceSchematic has a cache this would only care about spots where there was wood in your pattern.

So far I haven't found a use for as-long-as-it-is.

Apart from that, here's the current status:
I managed to make it not place blocks where a block on buildSkipBlocks belongs, but for whatever reason it still clears the spot and I haven't found the reason yet.

And for the highway I'd build the pattern out of wood, select it, make sure there's no unintended wood in my selection, turn on build repeat and use #sel replace wood obsidian. Because ReplaceSchematic has a cache this would only care about spots where there was wood in your pattern.

One extension to As-long-as-it-is proposed below.

OK, so, I am not understanding this...

I will explain better, and explicitly how I would use it on the highway. There are two processes, to dig the highway and to pave the highway. There are two ways to use as-long-as-it-is. First, I want my schematic to bridge, so, the layer below the obsidian layer, is included in the schematic. Since this is the road bed, so long as there is any competent building block there, I really do not care what it is. So my schematic is made of air and netherrack for the base. I can run this schematic with literal netherrack and it will bridge chasms and it will dig. But really, I do not care what the base it made out of, so long as it is solid. There were a lot of places where cobble or stone or granite or andesite or diorite had been used to make a place to walk, particularly in bridging over chasms. There were other things used that I really don't want in a road bed like dirt, leaf blocks, all sorts of crap blocks. Whatever my standards are, I have a list of acceptable blocks, and a list of unacceptable ones. I make the buildSkipBlocks set to was wood planks, build my road bed schematic out of wood planks and say in as-long-as-it-is the list of all the blocks I would be OK with, starting with netherrack -- if it is not OK, it is replaced by netherrack. Most importantly, if there is obsidian UNDER where the obsidian pavement will eventually go, that is OK, and I really do not want my bot to start uselessly mining obsidian that will be buried later, just to replace it with netherrack that will be buried later. That is one use, the sub-floor of the highway, be more forgiving, break and replace fewer blocks.

The second use in digging is to not mine out obsidian is the layer that will later be obsidian. So my schematic has a netherrack sub-layer, and a layer of buildSkipBlocks on the top in the area the obsidian will eventually go when the paving might happen. I set the as-long-as-it-is variable to Air Obsidian. The schematic will take down taller obsidian builds and leave them flush with the finished paved road bed, but will not mine any obsidian out in the digging stage that will just need to be replaced in the paving stage.

The third use I would use as-long-as-it-is, is in converting scarred terrain into natural terrain around something. I would list natural terrain in the as-long-as-it-is variable, probably with a grass block as the first block, but also include air, and then any block found that is not natural or Air is replaced by the first natural terrain block listed.

The fourth use I would use as-long-as-it-is blocks for is to eliminate caves in areas I do not intend to have air as part of the schematic but leave or make air where I intend for there to be air. I do not care what it is as long as it is a solid block. So the as-long-as-it-is block lists a lot of solid blocks I would be satisfied with.

I hope that at least one of these 4 examples proves the need for as-long-as-it-is.

EXTENSION TO AS-LONG-AS-IT-IS

In short, as-long-as-it-is becomes a build preference, the first element of the list being built if it exists in the hotbar (extended by allowinventory), if not then the next, etc, until we reach Air or the end of the list. If Air is not in the list, and none of the blocks are in the hotbar, then it allows baritone to fail in attempting top place the block in the world. Otherwise it places the first block in the list that it has stock of in the hotbar.

foreach (schematic block in the schematic) {
   if {schematic block == buildSkipBlock} {
     if {as-long-as-it-is is not set} continue;     <<< an error in my previous pseudocode
     if {world block in as-long-as-it-is} continue;         /* skip to next schematic block */

     set index 0;
     while {$index<[llength as-long-as-it-is} {
         set target world block = [ lindex as-long-as-it-is $index ];
         if {target world block==Air} { break; }
         if {target world block in hotbar} { break; }
         set index [expr $index+1];
      }
  } else {
     set target world block = schematic block;
  }
  try to place the target block in the world
}

I would use this extension with as-long-as-it-is to make one schematic for both paving and digging. The buildSkipBlock is set to Oak_Planks, and these are placed where the obsidian might be laid in the roadbed. The as-long-as-it-is is set to Obsidian Air -- obsidian first. So long as the player has obsidian, baritone lays it. When the player runs out of obsidian these blocks are still acceptable if there is already obsidian in the world, otherwise it makes the world block into air. It makes the Baritone bot resistant to failing. It paves when it can, and then just digs when it runs out of obsidian. It gets the most of out what the player has.

A railroad-road-bed grader type schematic, to make roads in the overworld would list all the potentially suitable blocks it might encounter in the overworld in the as-long-as-it-is list. As it progresses, it makes tunnels through hills and then reuses blocks to bridge ravine areas, providing a road bed for rails. It would only fail when it no longer has any acceptable blocks, and it has yet to reach the next hill to accumulate more. A superhighway grader schematic might do the same thing.

I didn't new people actually put road beds below their highways. A simple solution would be buildIgnoreExisting, but that wouldn't allow doing everything at once and would allow e.g. saplings in the a road bed.
You've found something I can't solve with the current settings, but I think you even mentioned a case where as-long-as-it-is is not enough: If I want it to put anything from a long list of solid blocks I've made in the bottom layer and obsidian or air in the top layer, I need to different sets of acceptable blocks while only having one as-long-as-it-is list.
Also for what you want we need more than just making more blocks valid, because even though it accepts more blocks as valid it will still only place the block listed in the schematic. (Everything I did so far only affects where the builder does stuff, not what it does)

I found the culprit for Baritone breaking blocks that should stay: They are defacto correct.
Because I don't have usable schematics lying around on my pc I used Baritones FillSchematic and ReplaceSchematic (which uses a FillSchematic when created via sel replace). I turns out that FillSchematic has some "smartness" itself and first wants air wherever there is a wrong block and only wants the target block when there's air, essentially replacing it. This causes buildSkipBlocks to correctly don't do anything (air shouldn't be skipped) until Baritone has broken the block where a skipped block was intended to be. Since Schematics loaded from files don't do this, they should already work.

You are correct, some of those things, if I combined them into one schematic would need separate buildskipBlocks and for each of them a different as-long-as-it-is list of blocks. To solve this problem I was just planning on making more schematics for different purposes. (I make a lot of schematics.) I would make one for rough roads people had cobbled together like within 10k of nether spawn in 2b2t. I would have another that handles the case that someone has dug the tunnel in the wrong geometry and we are not sure what might be under the rails further ahead -- there are a lot of non-standard digs in the ring roads. I would have yet another, the marathon schematic that runs until you run out of obsidian, and then keeps going just digging -- one would use these when far out and carrying a lot of road materials. It might run for hours AFK.

About the road base -- It is simply the blocks under the road, what is not dug out in the tunneling process when passing through void-free netherrack. By including these base blocks in the tunneling schematic, when baritone reaches a chasm, it will automatically take netherrack out of inventory and build the base for the road over the chasm. the chasm is bridged at dig time, it does not need to be bridged later. This way, when and if the pavers ever come through, they have an easy surface to lay obsidian on -- there is no risk of Baritone getting confused and thinking it has to scaffold or something. Also in between the time the road is dug, and it is paved -- which may be months -- if someone comes through on a horse, they have a workable road bed to ride on all the way out. Including the road base in the schematic also fixes little holes in the road and lava pockets in the road bed. If we ever run out of obsidian, the new highways may remain unpaved, so, a contiguous road bed is useful.

Does anyone know why this is in FillSchematic.java?

      } else if (current.getBlock() != Blocks.AIR) {
            return Blocks.AIR.getDefaultState();
      }

It makes it impossible for the builder to ignore specific blocks in the schematic.
I also tested it without these lines and didn't find any issues, but maybe there's a non-obvious reason.

Does anyone know why this is in FillSchematic.java

I ask that about a lot of baritone code

That is so that baritone makes sure to remove blocks before it places more in that place I believe, which is needed for obvious reasons.

The reason is not that obvious because BuilderProcess does that itself and Schematics loaded from files don't even have the possibility to dynamically change their content, so either there's another reason or I can remove it.

It is probably legacy code that is no longer needed anymore, so yes, you can probably remove it. If not leijurv will point it out when merging the pull if he ever checks this repo.

Well, another option is to label that section with comments that raise these questions, and then just copy and paste over the buildSkipblocks code to this module, duplicating it. As you said Zac, this code does not even run when people execute schematics from files, so, it is not really going to be a slowdown issue to have duplicate code.

How active is leijurv ?

How active is leijurv ?

lol

How active is leijurv ?

Well...

How active is leijurv ?

Just guess

Since I consider removing useless code as cleaner than duplicating code to places where it doesn't belong I'll just remove those 2 lines for now and if leijurv complains I can still use HoratioGamer's or similar solutions.

Is key'->'value[','key2'->'value2]... a good mapping syntax? With list-type values this is key'->'value11[','value12]...[','key'->'value21[','value22]...]... and list-type keys aren't possible because keys can't contain commas.

Don't worry about command parsing itself, Baritone's command system is fairly robust for what it does.

What is wrong with:

#set as-long-as-it-is "block_id1 block_id2 block_id3"

Just a space between block names... there is no key, they are all as long as it is blocks. Someone showed me a split... @living.... list = string.split(" ") I am guessing .split will do this. Because we just want to know if it is in the list, and preserve the order of the list.

Or are you trying to create multiple buildSkipBlocks, each with their own list of as-long-as-it-is block list? Where it is:
SkipBlock_id1->"as-long-as-it-is-list1" SkipBlock_id2->as-long-as-it-is-list2 ?

Like that is way complicated, and highly functional, and I would use it, but I am pretty sure I am the only one who would.

It's not about command parsing, it's about datatype parsing.
I can't use spaces because Baritone's command system uses them to split arguments and settings only accept one argument (that's why lists like acceptableThrowawayItems use "," and not ", " as a seperator)
Yes Horatio, it is a mapping of blocks to their skip blocks and I'm actually using split already.

(Everything I did so far only affects where the builder does stuff, not what it does)

Just found out that my thoughts on this were wrong. When checking block if a block is valid Baritone uses a valid(current,desired) method and I thought that's it and it only places what the schematic requests, but in addition it checks for every invalid block and for every item it is allowed to place if that item would place a valid block in that position, so valid() can indeed change what the builder does.
Is there a reason to suppress that behaviour for a buildValidSubstitutes setting or does everyone using it want Baritone to place the substitutes anyway? And should the block itself still be valid if it has substitutes specified but they don't contain the block itself (if it is not one can completely swap blocks using something like "stone->netherrack,netherrack->stone)"?

@HoratioGamer is it important that skip blocks aren't considered part of the schematic? With the solution I pushed to my repo (no pull yet, but feel free to use it) it just considers everything in their position as valid and thus penalizes breaking anything there while allowing to place blocks at 0 cost.
To treat them like any other block in the world I think I'd have to wrap the schematic with a new subclass of MaskSchematic, which is a lot more to do than the 10 (5 of them are javadoc) lines I already did.

I was holding off responding to your last post because I was uncertain and unknowledgeable.

I am not understanding because buildValidSubstitutes is not documented as far as I can tell. I have tried and it is not a command-line accessible variable as far as I can tell. I do not want to characterize it before I understand it. If you could give me a concrete example of how it gets set, or what values it might have or what influence it might have if I had a particular schematic....

For instance, the first example that springs to mind on if I lay two blocks from my inventory, and what appears in the world is different, is stairs doing the corner stairs with 3 accessible diagonals (outside corner) or 1 accessible diagonal (inside corner). I assume a schematic containing stairs would need to know it is a valid substitute to use regular linear stairs. My attempt at an example screenshot:

I understand the jist of this, there is this list buildValidSubstitutes for some reason I am not clear on. If I assume for a minute that the stairs thing would be covered in BuildValidSubstitutes, I am trying to understand, would this lead to a problem if part of the stairs were outside the schematic, or in the buildSkipBlocks part of the schematic.

I am going to venture a guess, if one were going to implement only buildSkipBlocks, then the most elegant solution be that these blocks are just not part of the schematic at all. This would mean, under no circumstances would baritone consider interacting with them as part of the build. I have observed baritone tunneling through blocks that are outside the schematic to build some part of a schematic, and most times I do not care. Sometimes when it does this tunneling behavior it is ruining the desired end effect -- to not have any holes in the walls.

However, if as-long-as-it-is logic is introduced, then these blocks are conditionally part of the schematic. I do not appreciate all the code problems this would bring.

buildValidSubstitutes sounds like As-long-as-it-s, sort of, and the answer might be to split buildSkipBlocks from as-long-as-it-is, and instead implement as-long-as-it-is as buildUserValidSubstitutes (copy code from buildValidSubstitutes, but use user-defined data) where the data might be a list like (Oak_Planks*, Obsidian, Air) meaning, the valid substitutes for oak planks in the schematic is obsidian or air in the world. The problem is, then there has to be a check if Obsidian is available in the hotbar (inventory) to avoid an error, and instead decide Air is the valid substitute that would be used.

I hate to split the two concepts, but, if it fits the code best, then until I learn JAVA, that is what it has to be....

As for the distinction you ask me to decide between in the latest message, I ask, may I test it ? See if it behaves well ? I have a problem that is kind of basic to Java. If I have multiple forge mods loaded, and more than one contains a copy of baritone, and I want to test your version of baritone in the presence of the balance of the other mods, how do I assure your baritone is the one that is being tested ? How do I make Minecraft/Forge ignore all the other copies of baritone ?

I have found your github, but, git-newbie question, how do I get a baritone.jar of this test code ? I see no downloads or packages that like, wrap it up as a product to download. I would like to avoid the whole download the entire repository and do this, this, and these really gitty things. It is an entire skillset that I would not get right on my first 20 tries.

* Edit: This will not work because oak_planks is not a valid designation. Only "planks" works, and then Baritone will not distinguish between planks in a schematic.
#2323 (comment)

OK, I did a little gitty, and found this....

https://github.com/ZacSharp/baritone/tree/buildSkipBlocks

Can you make this into a bartione_HG.jar for me ? And tell me, would it work with my schematica test configuration in MultiMC and forge making changes to the current mods as :

That is, I will remove baritone standalone and replace it with baritone_HG.jar, and I will remove World Edit and Optifine just to be careful. Would this be a viable test platform ?

If not, can you give me a description of how to install the baritone_HG.jar into Minecraft ?

Baritone penalizes breaking valid blocks

So, if I understand correctly, you are asking about other Baritone behaviour surrounding building the schematic, but not actually building the schematic. If a block were outside a prismatic schematic, I have not observed baritone either being careful of placing or breaking blocks. It is indifferent. Where there is a buildSkipBlock, it should be the same, it is simply not a part of the schematic that is built, as though it was not part of the schematic. If for some other reason it wants to build or break a block there, for the buildSkipBlocks without the implementation of the as-long-as-it-is feature, I do not care.

It is sounding more like, the as-long-as-it-is feature should be separate, perhaps buildConditionalBlocks and buildConditionalWhitelist.

I will test buildSkipBlocks soon and report on how it does.

Still don't have the feeling you understood what I meant.
What I'm talking about only affects pathing while building. Normally, Baritone slightly penalizes breaking and placing blocks as part of a path; while building breaking valid blocks (in this case including everything masked away by buildSkipBlocks) is even more penalized and placing valid blocks (again everything masked away) doesn't have any cost at all. This does in no way affect what Baritone thinks about a schematics completion or where it needs to place blocks.
You know how the pathing works, right?

I assumed it was a weighted A*, smallest metric is the path with the least penalty.

OK, so apart from not intentionally breaking or placing the blocks when considering the buildSkipBlock mapping to a point in the world, I think that Baritone should have the same metrics for non-schematic blocks as buildSkipBuild blocks. If they are in the way of pathing break them with the exact same penalty as blocks that were always outside the schematic. The I-do-not-care block is not a "Well, I would really prefer if were not broken even if it is in your way" block. It is "there is no schematic job to do here" block, anything else other than building schematics, I do not care.

If within the schematic there are weights, I am not seeing a logic to why right off.

No weighting outside the schematic, inside it weights based on whether a block is correct or not (break wrong block: x1, break correct block: xbreakCorrectBlockPenaltyMultiplier, place block where air should be: x2, place wrong block: x3, place correct block: x0). I can easily turn off the weighting for placing, by simply considering newly placed blocks invalid and I can turn off the extra breaking penalty in breakCostMultiplierAt by treating buildSkipBlocks IBlockStates as if they were null (not in schematic). Maybe I should do that in costOfPlacingAt as well.

Pushed a commit excluding them from the schematic. You can get it using git pull in your clone of my repo.

Next I want to do a setting to replace blocks with other blocks, e.g. for your long highway you could use namePending obsidian->obsidian,air,cobblestone->cobblestone,stone,netherrack,dirt to make the obsidian optional (it prefers blocks in the order they are listed in) and place some more blocks than just cobble where the schematic wants cobble.

Do you think there would be use for an additional setting that just changes which blocks are valid (the same as the above but without placing)?

(it prefers blocks in the order they are listed in)

OK, at a given block, anything in the list will do. If the block in the world does not match anything in the list, then check to see if we have the first block in the list in inventory, if so, lay that one, otherwise see if we have any of the next block in the list etc.

I think the above will satisfy those needs I currently anticipate.

Do you think there would be use for an additional setting that just changes which blocks are valid (the same as the above but without placing)?

This would be like a baritone checker bot.... when it comes upon a block that is not in the list, it what, stops ? There might be a use for that. Because if it is not going to place, and air is not the last thing in the list so breaking the current block to make air is not acceptable solution, then what else can it do but stop if it will not be placing ? It would be a way to implement a go-until-you-find blacklist by implementing it as a go so long as it is on the whitelist, which is the same as namePending cobblestone->cobblestone,stone,netherrack,dirt (but no place allowed)

Yeah, I just thought of a use.... a grief removing bot that does not break possibly historic signs. Go until it is not air, stop and let the operator decide. The problem is, the acceptable block is air, so, you have to not only not place, but also not break for that to be a use.

To get the jar use git clone https://github.com/ZacSharp/baritone.git, cd baritone, git checkout buildSkipBlocks and ./gradlew build, the output is the dist directory. Before doing the build itself you need to make sure you're using java 8, and if export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/ is not the correct way of doing it on your system (I use Linux) I can't help with that.
Your setup would be a viable test platform.

buildValidSubstitutes only exists in my workspace, not even in a local commit. I sure hope you can't find that.
It is meant as a mapping of blocks to a list of blocks Baritone considers valid in their position, which is as trivial as buildSkipBlocks if I don't want Baritone to consider the substitutes for placing. But since I want to add a way of changing the block palette of a schematic anyway, I'm wondering if there's any use to buildValidSubstitutes

I found a way to only penalize breaking where a block is skipped, so I can at least easily prevent it from wasting blocks. Would that be good enough? Just not breaking them, but placing like normal?

I found a way to only penalize breaking where a block is skipped, so I can at least easily prevent it from wasting blocks. Would that be good enough? Just not breaking them, but placing like normal?

OK, so, if I include Oak_Planks* in my schematic to mean, don't do anything at this block position, then it means do not do anything. Not breaking blocks saves me time and durability on tools. If the location is occupied by an Oak_Plank block, what exactly could it build there with only the buildSkipBlocks ? Block in the schematic is only a placeholder, I never want it built in the world. Baritone has a problem with building up against the nether ceiling. I want my buildSkipBlocks to go to the nether ceiling, if there is any chance they will be built as anything, it will totally screw up baritone -- it will start tunneling between the bedrock in the ceiling, stuff that would never come up anywhere but against the bedrock ceiling of the nether.

So, I believe the answer is no, there is no chance that just preventing breaking would be good enough, if the block is to be skipped, it cannot be built either.

* Edit: This will not work because oak_planks is not a valid designation. Only "planks" works, and then Baritone will not distinguish between planks in a schematic.
#2323 (comment)

Baritone penalizes breaking valid blocks and makes it free to place valid blocks as part of its path, meaning it could build bridge through a area where the schematic is filled with buildSkipBlocks with the same cost as if the blocks had already been there, essentially wasting blocks and maybe rarely time. It never tries to go there specifically to place buildSkipBlocks, it just favours placing scaffolding where the schematic contains buildSkipBlocks and avoids moving through (breaking) blocks where the schematic contains buildSkipBlocks.
I easily can turn the favouring for scaffolding off, but not the break penalty.

The straight line version for that for Ubuntu 18 is:

$ git clone https://github.com/ZacSharp/baritone.git
$ cd baritone
$ git checkout buildSkipBlocks
$ env | grep JAVA

Nothing found so I need to set JAVA_HOME

$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
$ env | grep JAVA

returns:

JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/

$ sudo apt install openjdk-8-jdk openjdk-8-jre
$ env | grep JRE

Nothing found so I need to set JRE_HOME

$ export JRE_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre

$ sudo update-alternatives --config java

There are 2 choices for the alternative java (providing /usr/bin/java).

Selection Path Priority Status

• 0 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1111 auto mode
  1 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1111 manual mode
  2 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java 1081 manual mode

Press to keep the current choice[*], or type selection number: >2

$ java -version
openjdk version "1.8.0_275"
OpenJDK Runtime Environment (build 1.8.0_275-8u275-b01-0ubuntu1~18.04-b01)
OpenJDK 64-Bit Server VM (build 25.275-b01, mixed mode)

finally....

$ ./gradlew build

(many signs of things compiling properly)

$ cd dist
baritone/dist$ ls
baritone-api-1.2.14.jar baritone-standalone-forge-1.2.14.jar
baritone-api-forge-1.2.14.jar baritone-unoptimized-1.2.14.jar
baritone-standalone-1.2.14.jar checksums.txt

run java -version

Did you start a new terminal before running java -version? Because that resets your env.
Same contents as my /usr/lib/jvm/java-8-openjdk-amd64/ directory.
By the way I never did any java installs to compile something, it simply worked when I tried it.

Report on Testing:

With the results of my first compile of baritone with buildSkipBlocks, I did ordinary baritone stuff, and there were no problems -- the null test.

As suggested, it git pulled and did a second compile. This is a summary of the results.

This was the very first schematic designed to use buildSkipBlocks:

It is a 2-advance long diagonal schematic. The intention is that the blocks next to the side wall remain in the schematic for one more advance so that in the presence of glitching, if a block next to the side wall glitches, there is an opportunity to correct it. This schematic tested perfectly in a creative world I have made modelling the nether highways of 2b2t at 1/100th scale.

I discovered, there is no functional #set buildSkipBlocks clear functionality:

so I #set buildSkipBlocks Obsidian because there is no obsidian in this schematic. I build the schematic without a functional buildSkipBlocks set, so that the sand was re-produced in my schematic workspace -- that worked correctly.

I then manually extended it to a 5-advance schematic with the buildSkipBlocks of Sand:

I then used this schematic in my model nether highways world

It worked perfectly.

The next test is to get it into 2b2t and test to verify it works there, and also, measure the effect on glitch blocks. So @ZacSharp @L1ving how do I get this to play happy with say this configuration:

Where Impact and KAMI/Blue both contain Baritone? You have not only added lines, you have removed lines, so if I drop baritone-standalone-forge in there, how would it know to use the version of the classes with the buildSkipBlocks in them ? Or do I have to unload all the other mods and go into 2b2t with a naked baritone ?

If your make sure that the name is alphabetically before the other two then it should work fine.

Where Impact and KAMI/Blue both contain Baritone?

Well you'll need to follow Impact's instructions for loading custom Baritone. KAMI Blue will use whichever Baritone is loaded in the classpath first, which means that with impact installed kami blue will use Impact's baritone

Sorry, forgot to add this to sort out impact, https://github.com/cabaletta/baritone/blob/master/SETUP.md#more-info this is some documentation about how to do that.

You should be able to clear it by resetting it (if I haven't messed up the default).
When using a mapping of valid but not placeable substitutes it won't fail when encountering them, as long as it has the block the schematic wants there.

void dealWithBlock(Block currentBlock, Block schematicBlock) {
    if (validSubstitutes.get(scheamticBlock).contains(currrentBlock)) {
        avoidBreaking();
    } else {
        normalBlockHandling(); // break if wrong, place schematicBlock if air
    }
}

this function doesn't exist ofc

What is the command to reset? Is it #reset buildSkipBlocks ?

yes, alternatively you can type more and use #set reset (#reset is an alias)

I think using valid substitutes to prevent it from breaking signs has two major flaws:

1. you would need to specify signs as a valid substitute for every single block the schematic contains
2. it may still break blocks supporting a sign, effectively breaking the sign as well

I think you shouldn't make air a valid substitute,

that would lead to Baritone only placing those blocks in order to walk on them or place blocks against them

the patch of cobble is built by hand to test if Baritone really doesn't touch it ("buildValidSubstitutes obsidian->air,stone->cobblestone,netherrack")

So it sounds like I can ditch Impact (I can live without Impact for testing, possibly forever), and just drop baritone-standalone in, and KAMI/Blue will automatically accept that baritone.... that was easy once you understand the magic ! Thank you all, and thank you to KAMI/Blue for just being smart about co-existing with others.

So then, winding back to your original question:

Do you think there would be use for an additional setting that just changes which blocks are valid (the same as the above but without placing)?

I guess the answer is no. That leaves:

Next I want to do a setting to replace blocks with other blocks, e.g. for your long
highway you could use namePending obsidian->obsidian,air,cobblestone->cobblestone,stone,netherrack,dirt
to make the obsidian optional (it prefers blocks in the order they are listed in) and place some
more blocks than just cobble where the schematic wants cobble.

That would meet all the needs I can anticipate.

Just to be clear, I could also say

#set namePending Oak_Planks*->Obsidian,Air

And that would implement the as-long-as-it-is functionality completely separately from buildSkipBlocks, which now is just a not-in-the-schematic marker.

This would allow me to do schematics that can do two different things at once -- one of the observations you had before, that I could only use i-do-not-care-blocks for one of the several applications at a time. With this I could do two. What incredible flexibility that would provide.

* Edit: This will not work because oak_planks is not a valid designation. Only "planks" works, and then Baritone will not distinguish between planks in a schematic.
#2323 (comment)

I think I found a rare use myself: when building something like a base partially intersecting with rocks you want it to leave the andesite where your schematic contains stone but obviously it shouldn't build the stone statue in the backyard out of andesite.

For buildSubstitutes (yes, named it) I ran into a little issue: I want it to prioritize placing the first blocks on the list but at the same time don't want it to break already placed correct blocks that are further behind. When just checking if the current block is on the list it obviously doesn't place anything if air is a substitute, but when first checking placeables and then the current block it will break correct blocks in order to replace them with higher priority blocks.
I guess the answer is accepting every correct block except air before checking placeables and then check if it is air and air is correct.

if (schematic block==buildSubstitutes.block_id) {
   if (world block in buildSubstitutes.list) {  skip this block like it is not part of the schematic; }

   set no_stock = 1;
   foreach {substitute in buildSubstitutes.list} {
      if {substitute == Air} {
        set desired world block = substitute; 
        set no_stock = 0;  
        break; 
      }
      if {substitute present in hotbar (inventory)} { 
        set desired world block = substitute; 
        set no_stock = 0;  
        break; 
      }
   }
   if {no_stock==1} { 
     throw baritone error "Unable to do, no stock of buildSubstitutes.list"
   }
} 

Put that right beside

if {schematic block == buildSkipBlock} { skip this block like it is not part of the schematic; }

and right before

make world block = desired world block

It seems everywhere you have put buildSkipBlocks code, is a place to put buildSubstitutes code too.

If buildSubstitutes was intended to be a list of elements of block_id->Sub1,sub2,sub3, describing multiple substitutes, then you would have to wrap the entire thing in another foreach substitute in buildSubstitutes.

It seems everywhere you have put buildSkipBlocks code, is a place to put buildSubstitutes code too.

Sadly only when dealing with logic based pseudocode. In reality it is a little more complex.

I'll quickly push what I used to get this and go to bed

buildSkipBlocks sand buildSubstitutes obsidian->obsidian,air,stone,stone,cobblestone,dirt,netherrackLittle tip (didn't try it myself yet, but I guess it will work): when doing the 2-task schematic, make the first one or two advances paving-only, so it has to place the road bed before the obsidian. Otherwise it will sometimes place the obsidian first and have a hard time reaching below if it's over water.

It's pushed. Tomorrow I'll try to preserve block properties when changing their type (e.g. preserve rotation when changing stair type) and allowing one to substitute specific blockstates only (e.g. flip all stairs upside down), don't know if I can do that though.

What you say is true, the orientation of things is a twist that is not included in my code, and in this case is not something that concerns me, however, in a different case, it came down to orientation here:

#2039

So, I understand, in other cases, I might find it important, and others might.

Thank you Zac

I definetly won't add the possibility to specify blockstates instead of blocks in the near future because that would require more complex parsers (blockstate string representation contains ",", so list and mapping can't simply split at "," anymore).
Preserving block properties when changing the block is still on my list, though. It is limited to SubstituteSchematic.
#2039 seems to be a little more difficult, unless there are no performance constraints for builder pathing (then I can simply do the blockstate prediction when checking if a block is inside the goal).
Nothing to thank for.

I have tested the second build a little more with the 5-advance diagonal schematic. I ran it down a tunnel that was already mainly dug, with a single-advance schematic. The schematic with buildSkipBlocks cleaned up all the glitches, but it did something odd. Ever few hundred blocks while it was running down a reasonably glitch-free section of highway, like 1024 and 512 exactly once each, but then other numbers that were not so obviously binary, baritone paused and was highlighting a path of blocks into the wall, and just stood still. It would then start moving again and continue cleaning up the glitches. I never noticed this when it was advancing slowly into the dig face. No other misbehaviour, but, I have not challenged it with an odd environment yet.

I have made the orthogonal diagonal schematic. Tomorrow I try it on another highway.

What you described fits my observations.
For me, when running to blocks out of render distance, it sometimes failed because it had no materials for the buildSkipBlock (sand in my case), which obviously shouldn't happen. I think what you saw is the same bug, but it somehow corrected itself fast enough to not fail.
The spots where it fails were all out of render distance when it started walking to them, right?

I managed to get some simple property copying (doesn't even work when target and source property have different possible values e.g. torch and end_rod) and noticed it is bad for highway building because stone, diorite, andesite and granite are all minecraft:stone, so Baritone can't build the variants when the variant property is preserved to be stone from the schematic.

The spots where it fails were all out of render distance when it started walking to them, right?

I will pay more close attention this time.

UPDATE: I have not observed it again. It could be as you say, a render distance thing. My seeing it at binary numbers, might just arise from chunks being 16 blocks wide -- it appeared in the chunk just outside render distance. This only caused a pause in the second tests, and, has not been observed while digging in the 3rd tests.

My second buildSkipBlocks schematic has worked perfectly in my 3rd round of testing. Again, my schematic only includes netherrack, air and sand -- the environment only netherrack, air, fire, lava, nether quartz and glowstone. It has not been severely tested. I should take my schematic to a desert with a shovel and see if it screws up with sand in the schematic and sand in the world.

Is there a buildSubstitutes modification I can try ? And if so, what do I give as the git command to get both the buildSkipblocks and buildSubstitutes code at the same time ?

$ git checkout buildSkipBlocks buildSubstitutes ????

or ?

And finally, what is the command line syntax for buildSubstitutes, and can you give an example of substituting Oak_planks to be Obsidian or Air ?

I pushed the buildSubstitutes to the buildSkipBlocks branch, so if your branch is up-to-date you already have them (if not do git pull)
The syntax is key->value,value,value,key->value,value,value,value,key->value,value, so basically everything before a -> is a key and the rest is comma separated values. For you example that would be #buildSubstitutes oak_planks->obsidian,air.

I have re-built baritone with both buildSubstitutes and buildSkipBlocks and will be testing schematics with those today.

When I copy and paste the text you have above:

oak_planks->obsidian,air

I can see it has no spaces, whereas the rendering in the code formatting makes it look like there are spaces after the commas ...

oak_planks->obsidian,air

Is it important there are no spaces ?

doesn't look like spaces to me

Google "kerning around commas". The render I see, it is either a lot of kerning and no space, or a reduced-sized space and no kerning. I illustrated the difference with my post.

The point is, I have had a lot of people ask me, is there a space there in some baritone commands, so when there are spaces, I put in a lot to be sure they see spaces, so there are simply no questions.

#build my.schematic    ~    118    ~

Well, again the render of github has betrayed me.... I had to put   to make that look right.

When there are no spaces, I ask if it is important. In retrospect if I had really understood this:

I can't use spaces because Baritone's command system uses them to split arguments and settings
only accept one argument (that's why lists like acceptableThrowawayItems use "," and not ", " as a
seperator)

I would have had my answer despite the ambiguity of the rendering.

Update on testing.... (still authoring)

So I have to report my first problems in making this work. I created the schematic pictured below. The intent was to use:

#set buildSkipBlocks Sand
#set buildSubstitutes obsidian->obsidian,air

#cancel
#build My_OA.schematic    ~-7    118    ~

where _OA means designed for use with Obsidian or Air. I forgot to run the #set buildSubstitutes command the first time, so the schematic was calling for obsidian for sure, and the schematic just did nothing. There was no warning about unable to do, nothing about not having the obsidian blocks needed, nothing. I ran my regular schematic that just uses the sand, and it ran. Then I realized that I had not set the buildSubstitutes variable, so, I did that as above and repeated the build....

(Note the torches above are not part of the schematic, they were added after for the screenshot.) Again, it did nothing:

Then I went back to my regular buildSkipBlocks schematic, it is working fine. So, I am not sure of the proper response when I try to build a schematic for which I do not have all the needed blocks at the first invocation, but I seem to remember when it has run properly, then you run out, it does give a message. When I did the buildSubstitutes, nothing changed -- it should have taken air as an acceptable substitute for the obsidian in the schematic above and done it as though there was no obsidian in the schematic. Like this:

It did not. There were no warnings, no comments nothing I can give you to help diagnose what is going on here. buildSubstitutes did not seem to function.

Sure

PMDB_Sand5_OA_zm7.zip
PMDB_Sand5_zm7.zip

Just rename them from .zip to .schematic.

Not that it is important, but the filename format I am using:

• PlusMinus heading,
• it is primarily a Digging schematic which automatically Bridges chasms
• Sand is the buildSkipBlocks,
• it covers 5 advances,
• z = ~-7 is the offset when facing +-, which means the offset is x = ~-7 when facing -+,
• where offset means, if you stand in the center of the highway this is the offset to make the schematic center on the center of the highway also
• _OA means Obsidian or Air ==> #buildSubstitutes obsidian->obsidian,air

Can you give me the commands to make a new branch for #2357 ?

if you wanna do it yourself, here you go:

1. git checkout buildSkipBlocks
2. git branch 'whatevernameyouwant'
3. git merge builderMaterialLogging
4. hope to not get conflicts / resolve them if they appear
   you are working in a clone of my repo, so there's nothing to do to get the branches

$ git checkout buildSkipBlocks
Already on 'buildSkipBlocks'
Your branch is up to date with 'origin/buildSkipBlocks'.

$ git branch 'Horatio_Gamer_Test'

$ git merge builderMaterialLogging
merge: builderMaterialLogging - not something we can merge

Did you mean this?
U

???? very gitty.

maybe try to checkout builderMaterialLogging once before doing it

$ git checkout builderMaterialLogging
Branch 'builderMaterialLogging' set up to track remote branch 'builderMaterialLogging' from 'origin'.
Switched to a new branch 'builderMaterialLogging'

$ git branch 'Horatio_Gamer_Test'
fatal: A branch named 'Horatio_Gamer_Test' already exists.

$ git merge builderMaterialLogging
Already up to date.

$ ./gradlew

$ mv baritone-standalone-forge-1.2.14.jar baritone-standalone-forge-1.2.14_HG4.jar

I will load it up before my next session.

uh, when you have run git branch Horatio_Gamer_Test once that branch already exists and you can check it out like any other branch

I have no precise idea what these commands do so, I am am just following instructions. Does this mean what I did will work -- _HG4 is viable for a test, or that it will not work, and I have to do it a different way ? With the queue to get into 2b2t, I ask instead trial and error.

You could give me the schematics so I can try reproducing it myself.
And if you want I can create a branch with the new builder settings and #2357, so you know what (and if?) blocks are missing when it fails. Wouldn't help when it just stalls though.

I just want to underline what a huge contribution buildSkipBlocks has been to the use of schematics for making highways in 2b2t. This is just one of the applications of using a schematic that does not completely fill a box. For illustration, this is what a single-advance schematic for baritone leaves behind when the server conditions are poor -- there should be a wide clear path down the middle.

Even though I have only been testing version 2 and 3 of the buildSkipBlocks code, and the 5-advance schematic I am using is likely overkill, it has completely erased this problem with no loss of Baritone performance. For comparison, this is what buildSkipBlocks leaves behind in identical server conditions (two different accounts digging on 2b2t at the same time using different mods for comparison):

Night and day, with the addition of a few lines of code.

Imma make the branch myself

Pushed it
it's git checkout horatioTesting or if that doesn't work git fetch git checkout horatioTesting (I think)

The problem is my new code, not what was already there. Baritone automatically marks blocks out of render distance as invalid (if they aren't known to be correct) and that's why it walks there at all. If I were to restrict the scanning radius around the player, even things like repairing a highway with nothing to do for one scan radius would break.

It does scan within builderTickScanRadius blocks every tick and does a full scan if there's nothing invalid it knows of, but if you have a lot of skipped blocks, they might be to far away for baritone to rescan them before breaking blocks to get there. So I have to prevent it from marking them as invalid.

I can confirm it targets skipped blocks outside render distance. I'm now thinking about whether I try to add a check to every spot were the content of the schematic is accessed without calling valid() or turn buildSkipBlocks into a wrapper schematic like buildSubstitutes and mapArtMode.

However I did not get any wrong behaviour with buildSubstitutes (it placed obsidian as long as possible and then continued without) and the only occasions where it stalled (eternal pathing) were when it could not reach a block, which is nothing new.

To prevent it from placing the obsidian before the netherrack without it breaking already present obsidian I successfully used a schematic where the first advance had gravel instead of obsidian with buildValidSubstitutes gravel->obsidian,air.

Sorry for being slow, I will be re-testing buildSubstitutes with the new build tonight.

About the render distance thing... is it not possible to just put a limit of how far away from the current position it checks ? I hate to say, make another variable ... I do not see how one will make the code happy to play on an art/show-piece server with really big baritone builds with a render distance of 16+ say, and 2b2t with a render distance of 6, and at the same time avoid the build function examining things outside render distance unless there is an adjustable limit beyond which it will not check. A buildCheckDistance variable? If > buildCheckDistance --> skip. If the schematic is contiguous (with build skip blocks, a schematic no longer needs to be contiguous), then in building some closer bit of the schematic, the previously skipped part is now within render distance. An example of a non-contiguous schematic I converted over from a 3d design file, a Colossus of Rhodes, broken off at the knees, reportedly like the original, to decorate a coastal city's harbor:

OK, so the question is, is the scan done frequently, or just once ? If it is done frequently, nothing will break if out of range blocks are scanned when they come in range on a subsequent scan. With the road schematics, something is happening where baritone is getting far ahead of the player's location. The schematic itself is small. There is nothing wrong with keeping baritone from buildrepeating too far ahead of the player. It just means it will scan again later.

The behaviour when it walks to blocks outside render distance seems to be normal Baritone behaviour, as it occured with a cleararea schematic, repeated along a long cleared path, shows it as well. However I still did something against marking skipped blocks that are out of render distance as invalid (pushed to horatioTesting).
I did not have any issues with it not starting without obsidian (had no obsidian during all tests) and I don't have any idea on why it might do nothing (not even eternal pathing) at all.

Ok, I have tested for the 4th time and now three times it started properly, only once did it seem to need obsidian... I will consider the first time might have been operator error and I will continue testing. In my next session I will update to the latest testing image.

When I did

$ git checkout horatioTesting

it did not say anything about downloading changed files from your git ?

At what stage would I see it downloading changed files ?

Success (mostly) buildSubstitutes is tested functional ! In total 4 tests were done. The first was a small test, just 40ish blocks of highway, but this was done with just one schematic using buildSubstitutes of obsidian->obsidian,air. It laid obsidian until it ran out, and then just dug which is the exactly correct behaviour. It switched over seamlessly, no hesitation whatsoever.

An important part of buildSubstitutes is that when it comes upon an acceptable block from the list of substitutes already existing in a correct position in the world, it is OK with that block and does nothing. The road building application of this is, when tunneling, and one comes upon obsidian that is correctly placed for the eventual road, it does nothing with it. This is very important because separate Dig or Pave schematics will do something undesired -- Dig schematics that need there to be Air at that location will uselessly mine out correctly placed Obsidian. Pave schematics will simply not dig if there is no obsidian in inventory. buildSubstitutes allows one schematic to both Dig and Pave, and revert to just digging when it runs out of obsidian, and when digging, never mines out correctly placed obsidian.

This is an essential feature of buildSubstitutes when using any schematic underground to ignore all of the variance that might be there, so long as the block is one of a select list of competent blocks. It would be important in making a grading schematic that makes railways and their beds in the overworld. On the subject of railway schematics, I envision two.... one is a tunnel and berm schematic, that makes a wide-base trapezoid cross-section berm under the rails when going through low areas, and an arched tunnel shape when going through hills. The second schematic would make instead a classic stone railway viaduct bridge with a repeated series of arches. The buildSubstitutes would make arches as tall as needed only to replace Air and incompetent blocks, sinking pillars to stone. It would leave Air around the bridge and in the vaults of the arches where it already exists -- two schematics to make the railway line seen in the first Harry Potter movie, or recreate some famous railway line. A long list of buildSubstitutes would make it possible to do landscaping on the slopes, among other things. buildSubstitutes has a lot of potential in any ground-engagement schematic. I am really excited about the potential of buildSubstitutes outside of building nether highways in 2b2t.

So I tested it by putting correctly placed obsidian in the road ahead. buildSubstitutes correctly did that too:

Here comes the mostly part. Unfortunately, the schematic refused to start running, unless one had a little obsidian on the hot bar. The picture above is from the second attempt. It did not start the first time because I had no obsidian. I put a little obsidian on the hotbar, it started up OK and once it was running, then when it ran out, it was OK it tunneled, and then when it found obsidian already laid ahead, but in the correct position for the eventual road, it just left it there and continued. 2 out of 3 ain't bad ! When I went to get a screenshot of it failing again, it did not fail... it started properly despite having no obsidian.

Finally, here is the schematic I was using:

I was using these settings:

;b set buildSkipBlocks sand
;b set buildSubstitutes obsidian->obsidian,air
;b buildrepeat 1,0,1
;b build PPDB_Sand5_OA.schematic ~ 118 ~

And this is the schematic, just rename it from .zip to .schematic:

PPDB_Sand5_OA.zip

Now, the remaining misbehaviours, buildSubstitutes not starting without some obsidian in the hotbar, and the tendency for buildSkipBlocks to pause to think about blocks that were originally outside render distance, these are not show-stoppers, so long as they are documented. The buildSubstitutes not starting could be very confusing to baritone users when there is buildSubstitutes set from the last session, and users forget, and they load up a schematic that is not intended to use buildSubstitutes, because even with the builderMaterialLogging set up, it did not give an explanation for why it was not starting, it just sat there. Again, I was unable to get a screenshot of a failure to prove this but I did get a screenshot of when, inexplicably to me, it did it properly in the 4th test.

Here is an example of the first of two railway schematics that would utilize both buildSkipBlocks and buildSubstitutes

the settings would be, and standing on the intended rail bed:

;b set buildSkipBlocks sand
;b set buildSubstitutes oak_wood_planks->cobblestone,stone,granite,diorite,andesite,air,spruce_wood_planks->dirt,stone,granite,diorite,andesite,cobblestone->cobblestone,stone,granite,diorite,andesite,dirt,dirt->dirt,cobblestone,stone,granite,diorite,andesite,gravel->gravel,cobblestone,stone,granite,diorite,andesite *
;b buildrepeat 3,0,0
;b build RR3.schematic ~ ~-4 ~-5 

* Edit: This will not work because oak_wood_planks is not a valid designation. Only "planks" works, and then Baritone will not distinguish between planks in a schematic.
This will not work because granite, diorite and andesite are not valid specifiers here.
#2323 (comment)

It builds berms with dirt sides that will grass-over in time when building in low areas, and will tunnel through hillsides. The order of appearance of dirt in the buildSubstitutes options lists is important -- we want to conserve dirt mined out to use it in the places were dirt is specified in the schematic -- to give grassy sides for our berm. This is the power of buildSubstitutes.

This would be the second schematic, to build a series of arches over ravines and oceans -- it is half-filled with sand so that one can see the structure that will be built, yet also see how sand fills the remainder of the bounding box. One would have to manually switch over from the tunneling and berming schematic to the bridging schematic:

;b set buildSkipBlocks sand
;b set buildSubstitutes stone->stone,cobblestone,granite,diorite,andesite,gravel->gravel,cobblestone,stone,granite,diorite,andesite *
;b buildrepeat 11,0,0
;b build RR_bridge11.schematic ~ ~-23 ~-2

* Edit: This will not work because granite, diorite and andesite are not valid specifiers here.
#2323 (comment)

Neither schematic uses any orientable blocks to make it more architectural, or landscaped, like stairs or flowers, and the actual rails are omitted because they are orientable too, and baritone sometimes does not do well with orientable things.

The commands you might need for testing:
git clone -> download repository, set it as remote "origin" and do some stuff making it easier to pull/push changes
git checkout -> switch to a (local) branch, doesn't download anything
git pull -> pull new commits from remote repo, that's were the downloads happen.
git fetch -> download commits but don't do more, needed to checkout branches that only exist on the remote so far
And the onthers I mentioned here:
git branch -> new branch from current
git merge -> merge a branch into the current branch

I got an error in the build, tried twice, decided to flush my directory and start over, still getting an error:

$ git clone https://github.com/ZacSharp/baritone.git

Cloning into 'baritone'...
remote: Enumerating objects: 35755, done.
remote: Total 35755 (delta 0), reused 0 (delta 0), pack-reused 35755
Receiving objects: 100% (35755/35755), 9.37 MiB | 1.74 MiB/s, done.
Resolving deltas: 100% (19548/19548), done.

$ cd baritone

$ git checkout horatioTesting

Branch 'horatioTesting' set up to track remote branch 'horatioTesting' from 'origin'.
Switched to a new branch 'horatioTesting'

$ ./gradlew build

Task :buildSrc:compileJava
Note: /home/username/Desktop/Minecraft/Baritone/baritone/buildSrc/src/main/java/baritone/gradle/task/ProguardTask.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

FAILURE: Build failed with an exception.

• What went wrong:
  A problem occurred configuring root project 'baritone'.

Could not resolve all artifacts for configuration ':classpath'.
Could not resolve org.spongepowered:mixingradle:0.6-SNAPSHOT.
Required by:
project :
> Could not resolve org.spongepowered:mixingradle:0.6-SNAPSHOT.
> Unable to load Maven meta-data from http://repo.spongepowered.org/maven/org/spongepowered/mixingradle/0.6-SNAPSHOT/maven-metadata.xml.
> Could not HEAD 'http://repo.spongepowered.org/maven/org/spongepowered/mixingradle/0.6-SNAPSHOT/maven-metadata.xml'. Received status code 520 from server: Origin Error

• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

BUILD FAILED in 1s

$

OK, is this me, or has some external source changed since I last tried to rebuild this ?

(yes, I deleted the entire local repository on my computer rm -R style, thinking I might have messed up some config file with a bad gitty command.)

This is an error happening in all projects using mixin's at the moment.

I got an error in the build, tried twice, decided to flush my directory and start over, still getting an error:

Flushing your cache was a bad idea. Mixin is offline, if you hadn't flushed your cache you could do ./gradlew --offline build

I think he flushed the repo, not the cache.

So is this a situation that is likely to correct itself ? I intend to recompile when I am able, move the new baritone to the mods folder, so it is in place before entering Q.

The latest pull fixes it. If you go into the build.gradle and change line 29 and 83 to :
url = 'https://repo.spongepowered.org/repository/maven-public/'

make sure it's https, not http

That worked thanks. I will do more testing with the latest version.

OK so I have to be clear.... buildSubstitutes is a mapping from schematic blocks to admissible world blocks.
buildValidSubstitutes is another mapping. Does one do its mappping first and the other second? So say that buildSubtitutes said oak_wood_planks->obsidian,air* and buildValidSubstitutes is obsidian->gravel,sand ... is Baritone intended to lay down gravel or sand in place of oak_wood_planks in the schematic ? The larger question is, what order are the mappings applied ? Can you give me an example that uses both ?

* Edit: This will not work because oak_wood_planks is not a valid designation. Only "planks" works, and then Baritone will not distinguish between planks in a schematic.
#2323 (comment)

Can you please test buildValidSubstitutes as well?

buildSubstitutes is applied before every other setting affecting the schematic, except for mapArtMode.
buildValidSubstitutes specifies blocks to ignore if they happen to be there (e.g. stone->gravel,sand will still only place stone, but will accept gravel and sand as well).
buildSubstitutes does almost the same, except that it will not accept the original block, unless it is listed explicitly and that it will consider all listed blocks for placing, prioritizing from start to end of the list (e.g. stone->gravel,sand will place gravel and sand, preferring gravel and accepting nothing else, unless changed by another setting)
I use it with a block I definitely can't place (e.g. torches and don't have torches) and set obsidian and air as valid substitutions, so it would keep obsidian that is there without placing new obsidian (can't use buildSubstitutes because torch->obsidian,air always prefers placing and torch->air,obsidian won't ever place anythin). Then I put torches instead of obsidian into the first row of a highway schematic, forcing baritone to always place the netherrack before the obsidian while still keeping already present obsidian.

In replacing buildValidSubstitutes with buildSubstitutes only works for optional blocks (i.e. they should be air unless there is something) and if there are no block state properties that can be copied. In my example it is possible and I'm not sure whether it will accept signs with any rotation in your example.
Two more examples where you have to use buildValidSubstitutes (I think) are allowing any meta of a block (e.g. stone->stone) and when you are building a house, you might want it to ignore the different underground materials but still build things like statues using the correct material (e.g. stone->dirt,obsidian,dirt->stone,obsidian,obsidian->dirt,stone in buildSubstitues would be awful, but in buildValidSubstitutes it's mostly fine).

There is no command that specifically clears them, but since they are empty by default you can just use #reset buildSkipBlocks, #reset buildValidSubstitutes and #reset buildSubstitutes

OK, so that is a valid use of both buildSkipBlocks and buildValidSubstitutes at the same time. It requires an area where there is no highway yet, and there is a void that needs bridging. I do not have one handy in 2b2t right now for testing (without making a highway to nowhere). So here is another idea: A highway repair and de-grief schematic. Use the buildSkipBlocks sand to keep it from worrying about areas beyond the road bed, but then use buildValidSubstitutes air->sign,banner* so it does not break signs and banners placed along the highway ? There are about 5.8 million blocks of existing highway I can use that schematic on. Am I understanding the logic correctly, that this is supposed to do will do what I think it will ? If so, this is a valid test of buildSkipBlocks and buildValidSubstitutes. I could then use buildSubstitutes air->air,sign,banner* to do the same thing with buildSubstitutes too.

Can you give me the literal command to clear these variables to be the empty value? I still do not know how to do that.

* Edit: this will not work because sign is wall_sign or standing_sign, and signs have rotation, only rotation 0 signs will match this specification.
#2323 (comment)