Packet issue with large modpacks including huge number of recipes?
chimericdream opened this issue · 30 comments
This was reported to me by a modpack creator trying to update my mod in their pack. After some digging, I was able to reproduce the issue both with and without my mod, so I don't believe the fault is on my side. However, the issue appears to be similar to one mentioned (and fixed) in Forge as well as an open bug in the Mojang bug tracker.
In short, although a server starts up fine, and players are able to load single-player worlds, they are unable to connect to a server. See the screenshot below:
Here are links to the bug (and fix) in Forge as well as the open ticket in the Mojang tracker:
MinecraftForge/MinecraftForge#6703
MinecraftForge/MinecraftForge#7485
https://bugs.mojang.com/browse/MC-185901
Is this something that makes sense to patch within Fabric itself in a similar fashion to Forge? If not, is there any recommendation I can take back to the modpack author other than "remove some mods"?
Thanks in advance!
MC 1.19.2
I am having this problem as well even with PacketFixer, has anyone found a cause or solution?
My modpack is also huge, and I have tested both hosting locally and in a good host with plenty resources.
The modpack was in the 500-600 mod ballpark when it was perfectly fine, but then I added some more and it grew to 700-800, which is when I starting facing that exact error "Cant fit X into 3".
I've been trying removing and adding mods from those I added, but it looks to either be cause by several (would be very unlucky of me if I happened to just add like 9+ incompatible mods all at once suddenly), or something related to a thing that grows as more mods are added, which some of the mods may add more of, causing it to go over a limit.
So I'm guessing it is related to recipes too.
There is a mod that "fixes" the mojang bug:
https://www.curseforge.com/minecraft/mc-mods/xl-packets-fabric
By "fixes" I mean it moves the goal posts to allow a larger number of recipes/blocks/etc to be sent over the network.
Can you provide a log? I'm not 100% sure this is caused by the packet size limitation?
I'm also experiencing this problem, but I don't know if it is connected to this:
Zh_mxl_G has disconnected, reasonInternal Exception: io.netty.handler.codec.DecoderException: java.lang.IllegalArgumentException: Payload may not be larger than 32767 bytes
Can you provide a log? I'm not 100% sure this is caused by the packet size limitation?
I can provide logs, getting home soon.
Payload may not be larger than 32767 bytes
Is the error I would have expected, the mod linked above should work around this.
The screenshot from @chimericdream doesnt show this, im not quite sure what its showing that error.
Im unsure if Fabric should start splitting all packets like forge is doing, it doesnt really seem like something that is in scope.
Payload may not be larger than 32767 bytes
Is the error I would have expected, the mod linked above should work around this.
The screenshot from @chimericdream doesnt show this, im not quite sure what its showing that error.
Im unsure if Fabric should start splitting all packets like forge is doing, it doesnt really seem like something that is in scope.
I'll test different situations and reply asap.
This mod doesn't solve the problem sadly :(
I think thats because you linked the wrong minecraft bug - this one looks more relevant?
https://bugs.mojang.com/browse/MC-117850
A quick search gives the scoreboard.dat being too large as a common cause of this error. e.g.
https://www.beastnode.com/forums/threads/disconnected.652/
https://board.aternos.org/thread/7863-cant-join-cant-fit-number-into-3/
but there are probably other causes?
Your 32767 error could be a problem with a long java.lang.String serialization?
See PacketByteBuf::DEFAULT_MAX_STRING_LENGTH=Short.MaxValue=32767
This issue does appear to be caused by a size limit, but it is in the net.minecraft.network.SizePrepender
Here it limits the size to a "VarInt" of length 3 - that is what the "unable to fit xxx into 3" means.
But the size of the VarInt needed to encode the buffer length for your packet is bigger than 3, this errors because it isn't supported.
Your error doesn't show what piece of code or packet is exceeding this limit, but you seem to believe it is the recipes or the
SynchronizeRecipesS2CPacket?
I am not entirely sure that this is a packet size issue. The Forge and Minecraft bugs I linked to came from this thread on the Forge forums. In it, the user mentions Forge #6703, which is what led me to this theory and the aforementioned bug in the Mojang tracker. As for the screenshot, that was originally from @TheButterbrotMan from the issue he posted in my mod.
Regarding the true actual cause, I am stumped. I did some digging through the Fabric source to see if I could find anything that looked like a plausible fix, but wasn't able to come up with a theory better than what I posted in the first comment. Since I could reproduce the issue both with and without my mod, I figured there might be some higher-level shenanigans going on that people here might have better theories about.
I think this problem could arise from many different situations (total packet size, list length overflowing, etc), so having different errors should not be a concern. I am unsure if packet splitting is a feature to put in FAPI, though - this may confuse proxies or already-existing mods. However, given the need to put the fix in both sides, it might make sense to use this as a server-side opt-in, client-side opt-out feature.
it might make sense to use this as a server-side opt-in, client-side opt-out feature.
Forge handles it based on the packet size.
If there are no large packets nothing changes. Which means it is easy for one side to be vanilla.
But when it spots a large packet, it triggers "tunnelling" of the big packet inside multiple custom packets, that are reassembled to the original packet on the other side.
Doing this doesn't cause a compatibility problem, since without this new behaviour it would just do the vanilla crash anyway.
Payload may not be larger than 32767 bytes
Server log: https://bytebin.lucko.me/GsALAaLQo4
I really would like if packet splitting could be a fabric feature, because I haven't found anyway to fix this problem.
Unable to fit:
Additional Infos to this problem are, that not all of my players experience this bug. Only the ones that tries to selfhost a server on the own computer, for example my serverpack does work on BisectHosting but not if selfhosted with Windows.
The pack we are talking about btw:
https://www.curseforge.com/minecraft/modpacks/deathdusk
So if you wanna test yourself I provide a Client- and Serverpack there.
@warjort The compatibility issue is not entirely about vanilla clients, although that is one thing. The major concern with this is that Fabric API replaces the vanilla behavior which mods and proxies might be relying on. For example, we can't reverse the order of loaded tags and recipes, since while the vanilla behavior is inconsistent (MC-226265) it is what the game is coded as.
That said, if this only triggers as a fallback path, it might make sense to put in the Fabric API. Not sure how that would work, though - overwriting SynchronizeRecipesS2CPacket is not an option we'd take if possible, as this is a compatibility risk.
@TheButterbrotMan Server-side DEBUG level log before/after the disconnect would be nice, since I believe it logs all packets.
@apple502j How can i get the debug log?
@TheButterbrotMan Server-side DEBUG level log before/after the disconnect would be nice, since I believe it logs all packets.
Here's an example log4j config file that is used by fabric loom when starting development environments.
https://github.com/FabricMC/fabric-loom/blob/dev/0.12/src/main/resources/log4j2.fabric.xml
It defines a debug log file.
To actually use it, copy it to your file system somewhere then define this java system property wherever you configure the jvm parameters to start the server
-Dlog4j.configurationFile=path/to/log4j2.fabric.xml
You will also need to change the line
<Logger level="${sys:fabric.log.level:-info}" name="net.minecraft"/>
to say debug instead of info for minecraft's logging (probably easier than explaining how to use the property it defines?).
Or you can just remove that line and it will use the Root logger rules.
See the following if you want to understand the details: https://logging.apache.org/log4j/2.x/manual/configuration.html
Ok Heyy ^^
@TheButterbrotMan asked me todo this with the debug.log and when i did it right, this is the debug.log
At 14:57:17 login message
and at 14:57:19 disconnect message
https://bytebin.lucko.me/5Y1sDTcwqS
Eventually a small and short feedback if i did it right :)
So this isn't the vanilla packet at all
Last vanilla packet here (2788 is the vanilla recipes)
[14:57:18] [Netty Server IO #1/DEBUG] (net.minecraft.class_2545) OUT: [PLAY:102] net.minecraft.class_2788
[14:57:18] [Server thread/INFO] (voicechat) Using Fabric Permissions API
[14:57:18] [Server thread/INFO] (net.minecraft.server.MinecraftServer) Marwin2001 joined the game
[14:57:18] [Server thread/INFO] (hyperlighting) Sending Config Packets to Marwin2001
[14:57:18] [Server thread/INFO] (STDOUT) [insane96mcp.progressivebosses.module.wither.feature.DifficultyFeature] [Progressive Bosses] Player Marwin2001 spawned a Wither. Difficulty: 0
[14:57:19] [Server thread/DEBUG] (FTB Quests) Wrote 35082 bytes, 604 objects
[14:57:19] [Server thread/INFO] (net.minecraft.class_3324) Marwin2001 uses DualWielding Mod
Nbt crafting packet here which fails:
[14:57:19] [Server thread/INFO] (de.siphalor.nbtcrafting.NbtCrafting) [Nbt Crafting] Syncing advanced recipe data to player Marwin2001
[14:57:19] [Netty Server IO #1/DEBUG] (net.minecraft.class_2535) Failed to sent packet
io.netty.handler.codec.EncoderException: java.lang.IllegalArgumentException: unable to fit 2121510 into 3
It is the "nbt crafting advanced recipe" packet that is too big
https://github.com/Siphalor/nbt-crafting/blob/f0266e6d3b011fa82491d40bf113a025e76c3e69/src/main/java/de/siphalor/nbtcrafting/mixin/network/MixinPlayerManager.java#L67
You can report this to that project. They will need to workaround the 2M (21 bit - VarInt3) packet size limit.
You may want to leave this open as a feature request of generic handling of big packets by the fabric-api?
Ehm. I removed NBT-Crafting and the error still occurs.
https://bytebin.lucko.me/AiyEq1mORZ
What now?
Do you have the debug log?
However, for the last debug log I only identified nbt crafting because it did its own logging to say it was sending a packet.
There is no logging from fabric or minecraft that says what non-minecraft packets it is sending.
So if there is another packet AFTER the nbt crafting packet that is the real cause of the problem, we would not know what it is from the log.
Ok hey its me agin... :)
This is now the debug log without NBT-Crafting
https://bytebin.lucko.me/4IjKfQE0E7
As I predicted there is nothing in the log to say which packet is causing the problem.
It does look like a login packet for one of your mods.
Looking at your mod list you have mods like badpackets (used by wthit) and packet_tweaker that mess around with how minecraft handles packets. Looking at github, packet_tweaker is no longer used by server_translations in its latest (1.19) version. There maybe other mods (e.g. packet size doubler?) doing funny things, but you have 587, it would be impossible to audit them all.
You need someone who can reproduce this problem to debug it.
I tried downloading the modpack, but it crashed during load with some error which I think was it trying to modify my taskbar with a native call to windows? I don't even want to think about that. :-)
As I predicted there is nothing in the log to say which packet is causing the problem. It does look like a login packet for one of your mods.
Looking at your mod list you have mods like badpackets (used by wthit) and packet_tweaker that mess around with how minecraft handles packets. Looking at github, packet_tweaker is no longer used by server_translations in its latest (1.19) version. There maybe other mods (e.g. packet size doubler?) doing funny things, but you have 587, it would be impossible to audit them all.
You need someone who can reproduce this problem to debug it. I tried downloading the modpack, but it crashed during load with some error which I think was it trying to modify my taskbar with a native call to windows? I don't even want to think about that. :-)
Youre on Linux?
Windows 11, but if you have mod trying to use sun jna (java's native call interface) I wouldn't touch it with a very long stick :-)
I am a suspicious person. If a mod is using jna it could be doing anything.
Even if it is legit, I don't want it messing around with my desktop.
In this case, it found out it didn't have permission and crashed, but what else might it change that I haven't got locked down properly? Maybe it will install a cryptominer? :-)
Hey I am running into the same issue (couldn't find forge github new to this). My servers log only shows this
[19:18:51] [User Authenticator #4/INFO] [minecraft/ServerLoginPacketListenerImpl]: UUID of player Pizarini is 8925946d-b53e-4ebf-bece-9da0cc07fe2d
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 0
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 1
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 2
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 3
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 4
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 5
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 6
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 7
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 8
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 9
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 10
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 11
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 12
[19:18:52] [Netty Epoll Server IO #5/INFO] [Calio/]: Received acknowledgment for login packet with id 13
[19:18:55] [Server thread/INFO] [minecraft/ServerLoginPacketListenerImpl]: com.mojang.authlib.GameProfile@227a2455[id=8925946d-b53e-4ebf-bece-9da0cc07fe2d,name=Pizarini,properties={textures=[com.mojang.authlib.properties.Property@2c7858d1]},legacy=false] (/31.205.120.105:56546) lost connection: Disconnected
I run a modpack im trying to create called. "Pizarro Industrial Magic". Any help would be much appreciated, it works in single player.