Describe the bug

I found this in atm 10 if I try to autocraft a 1048m storage disk the screen is loading endless sometimes. Sometimes the autocraft loads but at least a feedback would be nice like a loading bar. This would help to see if the autocrafting still loading or stuck.

How can we reproduce this bug or crash?

1. go on a atm10 server
2. create a refined storage system with the patterns for the 1048m storage disk and the items needed
3. try to start the autocrafting for the disk

What Minecraft version is this happening on?

Minecraft 1.21.1

What NeoForge or Fabric version is this happening on?

NeoForge for 1.21

What Refined Storage version is this happening on?

Refined Storage 2.0.0-beta.2

Relevant log output

Somebody posted this on Discord:

• https://mclo.gs/88ZkTEM
• https://mclo.gs/kzRPJKe

Hey! Just adding a bit more info I noticed while I was playing.

It seems this issue happens when there are missing resources to craft the item. There also seems to be a correlation where the more items are missing, the longer it takes to calculate the required resources.

For example, when trying to request a 16384k Storage Disk (ExtraStorage), around 122 blocks of Bio Fuel are needed (used to craft slimes via Mekanism infusing, which are then used for processor bindings, etc.). If all other required resources are present along with those 122 Bio Fuel blocks, the calculation takes about 1 second or less.

But if I remove just 1 Bio Fuel block (leaving 121 in storage), the calculation already takes around 5 seconds. Removing 32 blocks bumps it up to about 25 seconds. And if all Bio Fuel is missing, it takes around 1 minute and 30 seconds to finish calculating.

Not sure if this is the only cause of the slow crafting calculation, this is just what I’ve noticed so far. Hope it helps!

I am also experiencing this bug on atm10 4.2 (Neoforge 21.1.185). Specifically autocrafting large storage parts. It calculates just fine for the middle size disks (16k), but hangs for anything 64k or larger. When I disconnect my machine autocrafters (smelting for glass) it can go up to 256k before hanging, but it hangs no matter what for everything above 256k. The longest I let it go was 60 seconds before giving up.

Some things I'm noticing. Indeed, autocrafting large items takes a long time, especially if there are missing items.

The reason why it sometimes just "stops" working is that people keep retrying, and autocrafting has a maximum of 4 threads available. So if you keep and keep and keep retrying... you are saturating the thread pool and you need to wait for a response until all your scheduled requests are processed.

So would the fix be to stop the calculation if the screen gets closed, or add some kind of cancel button?

I think 4 changes are required:

1. The autocrafting algorithm should be more performant on large complex crafts (if possible).

2. Pressing cancel (or closing the preview in any way) should stop all pending requests.

3. There should be a watcher killing pending tasks after 5 seconds or so anyway.

4. Block requesting previews if no threads are available. You shouldn't be able to queue requests forever.

Since Thread.stop is discouraged in Java we should use the cooperative way and check if the current thread is interrupted and break out of the crafting tree recursion.

I am also experiencing this bug. I would be happy to PR a fix if needed. @raoulvdberge

(or provide world file, logs etc)

It's being worked on, be patient

Fixes:

• Closing the Crafting Preview window (via cancel or escape) will now properly cancel pending calculations.
• Autocrafting requests that take too long will be automatically stopped after 5 seconds.
• It is no longer possible to queue an unlimited amount of autocrafting requests when the autocrafting system is busy.

So, with these fixes the autocrafting system should remain available when there are lots of slow requests going on.

Let's talk about performance. Indeed, autocrafting calculations are slow on very large, complex, crafts.
An example is crafting the 1048m disk from Extra Storage, which doesn't even complete at this point.
But why?

The autocrafting system in RS2 is exhaustive in finding possibilities for your crafting request.

• A crafting request has X subcrafts.
• Each subcraft has Y ingredients.
• Each ingredient has Z possibilities.

The autocrafting algorithm in RS2 will check all the possibilities.
If you have enough resources for your craft, this will typically be fast, since not many variations have to be checked.
But if you end up with missing resources, it could be possible that the algorithm has to backtrack N levels back and retry a next possibility.
On a large scale, this becomes very slow. Especially if you have an empty network (everything is missing).

But it also has it's advantages, let me illustrate with following examples.

Look at following example, in Refined Storage 1.x:

• The system knows how to craft wooden planks, spruce planks, sticks and tripwire hooks.
• All patterns use Exact Mode OFF, meaning they should accept substitutions.
• The patterns for tripwire hooks and sticks use regular wooden planks.

This is the state of our Refined Storage 1.x system:

We don't have regular wooden planks or logs, but we have spruce logs.
If we now request a tripwire hook, the craft fails:

Why?

• The autocrafting process was not exhaustive in finding possibilities.
• For subcrafts, it's not checking all substitutions.

Let's take a look at Applied Energistics 2.

The same patterns that were stored in Refined Storage 1.x:

• The system knows how to craft wooden planks, spruce planks, sticks and tripwire hooks.
• All patterns allow substitutions.
• The patterns for tripwire hooks and sticks use regular wooden planks.

This is the network state:

And when we request a tripwire hook:

The same problem, it insists on using oak planks.

Now, let's look at Refined Storage 2.

All the same patterns with fuzzy mode "ON":

This is the network state:

And if we request a tripwire hook:

This autocraft DOES work, as opposed to Refined Storage 1.x and AE2.

Refined Storage was smart enough to use the spruce planks.

Will I be "fixing" autocrafting performance for very large crafts?

Not at this time - and here’s why:

Yes, it's true that requesting very large recipes isn't ideal right now. But in practice:

1. These massive crafts are rare, and when needed, you can split them into smaller sub-crafts fairly easily.
2. Normal crafting requests perform just fine. No slowdowns.
3. Even large crafts perform fine, but it goes wrong when you have lots of missing items, as it needs to do more checks then.
4. For standard usage, this new autocrafting system shines: it thoroughly checks ingredients and handles dependencies better than ever.
5. The code changes linked in this issue will ensure that the autocrafting system will remain available even when the system is under heavy load due to lots of large tasks.

We're currently in Beta 2, and this new autocrafting system is designed to be powerful and flexible.

It's difficult to fix performance and not water down its capabilities.

Let's give it some time in the wild and see how it plays out before making big changes.

I think your justification makes sense. I would love to see a UI msg for when crafts are too complex for RS2 and some suggestion on how to get the autocraft completed. Based on your comment it sounds like the right verbiage is "Too many subcomponents are required. First craft some of the subcomponents and try again."

I've added a UI message with my latest changes:

This request took too long to calculate, and was cancelled.
Try crafting a smaller amount in multiple steps.

I think your text is better, though. I'll change it to:

This request took too long to calculate, and was cancelled.
Too many subcomponents are required. First craft some of the subcomponents and try again.

Hi, when playing i grew increasingly frustrated about the autocrafter not being able to calculate items. I modified the code, recompiled it and tested it out on my atm10 installation. Just by changing the timeout of the autocrafter, helped a lot. This might be unrealistic, but i changed the timeout to 500 seconds (i really didnt mind waiting for a large item to be calculated for that long). Suprisingly enough, that fixed the issue, even the infinite storage part is being calculated now. For some reason, also the speed of the calculation increased, even calculating larger storage parts (16k) is for one; possible and also relatively quick, under 2-3 seconds. This shouldnt make sense, but that are the symptoms Im experiencing.

Keep in mind that i have a minimal amount of experience with this codebase, i just played around with the code for 30 minutes, so these insights might not be worth much, but might serve as a lead for further experimentation.
Specifically i changed TIMEOUT_MS to be 5000000 instead of 5000 in ../refinedstorage2/refinedstorage-common/src/main/java/com/refinedmods/refinedstorage/common/autocrafting/TimeoutableCancellationToken.java

Even though your explanation for the lack of performance is satisfactory @raoulvdberge , I was clearly way too lazy to craft 27 1k storage parts 27 times (that again had to be broken down into subcomponents) in order for the crafter to maybe craft a infinite storage part. The main reason you set up autocrafting, is to not have to perform repetetive crafts over and over again.

Oh and by the way, im running this on Neoforge v. 21.1.206, in a modpack; atm10 to the sky, i modified this mod v. 2.0.10 beta, minecraft v. 1.21.1, JWJGL v. 3.3.3

@KacperRzadkowski
The reason calculating 16k storage parts got faster is because atm10 sky contains an old version of RS (beta.10) and recompiling the code automatically updated it to the latest version (beta.11) which contains autocrafting performance improvements

The improvements are part of beta 12, which is not released yet.