Performance of Compacting Drawers findAllMatchingRecipes

Question

Performance of Compacting Drawers findAllMatchingRecipes

metalshark opened this issue 3 years ago · 6 comments

metalshark commented 3 years ago

The findAllMatchingRecipes routine can cause server watchdog timeouts, at scale with a high influx of items each tick. Can the results of it for a given CraftingInventory be cached to reduce the work required each time it is called? An example crash:

java.lang.Error: ServerHangWatchdog detected that a single server tick took 60.09 seconds (should be max 0.05)
	at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.<init>(Int2IntOpenHashMap.java:101) ~[?:?] {re:classloading}
	at it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap.<init>(Int2IntOpenHashMap.java:122) ~[?:?] {re:classloading}
	at net.minecraft.item.crafting.RecipeItemHelper.<init>(SourceFile:23) ~[?:?] {re:classloading}
	at net.minecraft.item.crafting.ShapelessRecipe.func_77569_a(ShapelessRecipe.java:51) ~[?:?] {re:classloading,pl:accesstransformer:B}
	at net.minecraft.item.crafting.ShapelessRecipe.func_77569_a(ShapelessRecipe.java:15) ~[?:?] {re:classloading,pl:accesstransformer:B}
	at net.minecraft.item.crafting.IRecipeType.func_222148_a(SourceFile:30) ~[?:?] {re:classloading,re:mixin}
	at net.minecraft.item.crafting.RecipeManager.func_215380_a_(RecipeManager.java:94) ~[?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:crumbs.mixins.json:RecipeManagerMixin,pl:mixin:A,pl:runtimedistcleaner:A}
	at net.minecraft.item.crafting.RecipeManager$$Lambda$15318/1364943273.apply(Unknown Source) ~[?:?] {}
	at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0_201] {}
	at java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1628) ~[?:1.8.0_201] {}
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_201] {}
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_201] {}
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_201] {}
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_201] {}
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_201] {}
	at net.minecraft.item.crafting.RecipeManager.func_215370_b(RecipeManager.java:97) ~[?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:crumbs.mixins.json:RecipeManagerMixin,pl:mixin:A,pl:runtimedistcleaner:A}
	at com.jaquadro.minecraft.storagedrawers.util.CompactingHelper.findAllMatchingRecipes(CompactingHelper.java:171) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.jaquadro.minecraft.storagedrawers.util.CompactingHelper.findHigherTier(CompactingHelper.java:79) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.jaquadro.minecraft.storagedrawers.block.tile.tiledata.FractionalDrawerGroup$FractionalStorage.populateSlots(FractionalDrawerGroup.java:432) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.jaquadro.minecraft.storagedrawers.block.tile.tiledata.FractionalDrawerGroup$FractionalStorage.setStoredItem(FractionalDrawerGroup.java:197) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.jaquadro.minecraft.storagedrawers.block.tile.tiledata.FractionalDrawerGroup$FractionalDrawer.setStoredItem(FractionalDrawerGroup.java:596) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.jaquadro.minecraft.storagedrawers.capabilities.DrawerItemRepository.insertItem(DrawerItemRepository.java:53) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.jaquadro.minecraft.storagedrawers.api.capabilities.IItemRepository.insertItem(IItemRepository.java:43) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.jaquadro.minecraft.storagedrawers.capabilities.DrawerItemHandler.insertItemFullScan(DrawerItemHandler.java:89) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.jaquadro.minecraft.storagedrawers.capabilities.DrawerItemHandler.insertItem(DrawerItemHandler.java:62) ~[storagedrawers:1.16.3-8.3.0] {re:classloading}
	at com.lothrazar.cyclic.base.TileEntityBase.moveItems(TileEntityBase.java:312) ~[cyclic:1.16.5-1.5.7] {re:classloading}
	at com.lothrazar.cyclic.base.TileEntityBase.moveItems(TileEntityBase.java:286) ~[cyclic:1.16.5-1.5.7] {re:classloading}
	at com.lothrazar.cyclic.block.cable.item.TileCableItem.normalFlow(TileCableItem.java:92) ~[cyclic:1.16.5-1.5.7] {re:classloading}
	at com.lothrazar.cyclic.block.cable.item.TileCableItem.func_73660_a(TileCableItem.java:69) ~[cyclic:1.16.5-1.5.7] {re:classloading}
	at net.minecraft.world.World.func_217391_K(World.java:491) ~[?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:runtimedistcleaner:A}
	at net.minecraft.world.server.ServerWorld.func_72835_b(ServerWorld.java:371) ~[?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:mixins.stevekung's_lib.json:server.level.MixinServerLevel,pl:mixin:A,pl:runtimedistcleaner:A}
	at net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:851) ~[?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:globaldataandresourcepacks.mixins.json:MinecraftServerMixin,pl:mixin:A,pl:runtimedistcleaner:A}
	at net.minecraft.server.dedicated.DedicatedServer.func_71190_q(DedicatedServer.java:291) ~[?:?] {re:classloading,pl:accesstransformer:B}
	at net.minecraft.server.MinecraftServer.func_71217_p(MinecraftServer.java:787) ~[?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:globaldataandresourcepacks.mixins.json:MinecraftServerMixin,pl:mixin:A,pl:runtimedistcleaner:A}
	at net.minecraft.server.MinecraftServer.func_240802_v_(MinecraftServer.java:642) ~[?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:globaldataandresourcepacks.mixins.json:MinecraftServerMixin,pl:mixin:A,pl:runtimedistcleaner:A}
	at net.minecraft.server.MinecraftServer.func_240783_a_(MinecraftServer.java:232) ~[?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:globaldataandresourcepacks.mixins.json:MinecraftServerMixin,pl:mixin:A,pl:runtimedistcleaner:A}
	at net.minecraft.server.MinecraftServer$$Lambda$14313/554322589.run(Unknown Source) ~[?:?] {}
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201] {}

metalshark · Answer 1 · 2021-12-21T05:47:42.000Z

Used to happen once a day at least before deploying the patch a while back, now the server hasn’t fallen over since. There are a huge number of recipes (playing with SkyFactoryOne). However the problem seems to scale with the number of compacting drawers receiving output from automated systems, such as Cobble4Days and using that to get ExCompressum 9x compressed cobblestone in a line. So when several players have several compacting drawers receiving output or being used as input every tick it soon adds up and the server tanks.

…

On Mon, 20 Dec 2021 at 22:55, Justin Aquadro ***@***.***> wrote: How often does this show up? Is the 60 seconds reported from a single call to the method? And how big would you estimate your recipe list is? — Reply to this email directly, view it on GitHub <#975 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABD2FA5HYGIXEAEMVEZTYTUR6X7ZANCNFSM5IGSHHLQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

jaquadro · Answer 2 · 2021-12-20T22:55:46.000Z

How often does this show up? Is the 60 seconds reported from a single call to the method? And how big would you estimate your recipe list is?

jaquadro · Answer 3 · 2021-12-21T06:28:22.000Z

Do you know if things were setup in a way that they would empty out and clear, then re-populate? Once it's populated it shouldn't need to call that method again. Should be true even for leaving/joining world.

metalshark · Answer 4 · 2021-12-21T06:31:12.000Z

Yes. So you’d have the cobblestone generators going through the tiers up to 9x, when the top tier was reached for a compactingdrawer it would be emptied, then filled again. The same with things like slime into slime blocks with the slime blocks then being pulled out (emptying the compacting drawers). About half of them would be emptied almost every second.

…

On Tue, 21 Dec 2021 at 06:28, Justin Aquadro ***@***.***> wrote: Do you know if things were setup in a way that they would empty out and clear, then re-populate? Once it's populated it shouldn't need to call that method again. Should be true even for leaving/joining world. — Reply to this email directly, view it on GitHub <#975 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABD2FAG4CZOWVLU4S424NLUSANBBANCNFSM5IGSHHLQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

jaquadro · Answer 5 · 2021-12-25T13:43:05.000Z

I've put in a limited solution that I think will solve the majority of the issue, by having the compacting drawers remember what they previously held and what item was originally inserted. This will avoid the expensive population call if they're emptied and re-filled with the same item.

jaquadro · Answer 6 · 2021-12-26T18:14:15.000Z

This is now in the latest 1.16

Share to