Server crash
rrichardson-uw opened this issue · 24 comments
Hey,
thanks for fixing the other one, I updated to 4.7.4 and everything seemed fine, but just got this:
Just turned on debug mode and got another crash, so this one includes that too:
Thanks again. :)
Hey,
I am using paper spigot.
I have other plugins on the server that also use it though, and have no issues with them.
Also, is protocolLib integration a new thing for mobhunting? As I've been running the plugin for months now and it has been fine prior to these recent updates, but I can't say I've been keeping track of if it was implemented prior.
I will disable the integration for now though and let you know how that works out.
It looks like you are using paper spigot?
Maybe protocollib and paper is not 100% supported.
You can try disable integration to protocollib in mobhuntings config.yml
The integration is not new, but I started sending a new type of packets a few days ago. I have added Farm detection and blacklistarea to the plugin. And when a blacklisted area is found i show the area with a circle of flames around the area.
https://media-elerium.cursecdn.com/attachments/201/138/2017-04-02_19.png
I can see from the second log you pasted that the crash happened after the plugin found a black listed area and that sendPacket is involved. So I guess it the new feature which caused the crash.
I cant tell if it protollib which is the problem or if i send to many packets to fast.
Hey,
Disabling ProtocolLib does appear to have stopped the crashes, but I'm not sure it's the cause, rather it has just prevented the situation happening? (I assume disabling it stops the torches?) I have at least 3 other plugins on the server that use it with no issue so far.
Is it possible the problem comes from the location being detected twice at the exact same time by the same check? Take this below:
[22:16:48] [Thread-109/INFO]: [MobHunting][Debug] ======================== New kill ==========================
[22:16:48] [Thread-109/INFO]: [MobHunting][Debug] MrSuperRed killed a WitherSkeleton (Minecraft)
[22:16:48] [Thread-109/INFO]: [MobHunting][Debug] Permission 'mobhunting.mobs.*' or 'mobhunting.mobs.WITHER_SKELETON' not set, defaulting to True.
[22:16:48] [Thread-109/INFO]: [MobHunting][Debug] Basic Prize=F4 for killing a WitherSkeleton
[22:16:48] [Thread-109/INFO]: [MobHunting][Debug] Found a blacklisted grinding area = Location{world=CraftWorld{name=helloclan_nether},x=5254.110059600447,y=67.00873057063264,z=-408.92500001192093,pitch=10.0,yaw=143.65648}, range=50.0
[22:16:48] [Thread-109/INFO]: [MobHunting][Debug] ======================= kill ended =========================
There was no crash when that happened. But this one:
[22:19:48] [Thread-109/INFO]: [MobHunting][Debug] ======================== New kill ==========================
[22:19:48] [Thread-109/INFO]: [MobHunting][Debug] TheGamingNerd_ killed a Spider (Minecraft)
[22:19:48] [Thread-109/INFO]: [MobHunting][Debug] Permission 'mobhunting.mobs.*' or 'mobhunting.mobs.SPIDER' not set, defaulting to True.
[22:19:48] [Thread-109/INFO]: [MobHunting][Debug] Basic Prize=F6 for killing a Spider
[22:19:48] [Thread-109/INFO]: [MobHunting][Debug] Found a blacklisted grinding area = Location{world=CraftWorld{name=helloclan},x=167.9487068986283,y=67.0,z=-377.1004021703889,pitch=-16.060541,yaw=359.05228}, range=50.0
[22:19:48] [Thread-109/INFO]: [MobHunting][Debug] Found a blacklisted grinding area = Location{world=CraftWorld{name=helloclan},x=167.9487068986283,y=67.0,z=-377.1004021703889,pitch=-16.060541,yaw=359.05228}, range=50.0
[22:19:48] [Thread-109/INFO]: [MobHunting][Debug] ======================= kill ended =========================
There was when this happened, and as you can see it located the same place twice, in the same debug section (classing debug section as any debug info within the same start/end block i.e. "new kill" and "kill ended"), so I assume it would have wanted to make 2 sets of torches occur in the exact same place simultaneously?
I've noticed there are other times when it detects the same area twice, but those seem to be accompanied by different errors:
[21:07:45] [Server thread/ERROR]: Could not pass event EntityDeathEvent to MobHunting v4.7.2
org.bukkit.event.EventException
at com.destroystokyo.paper.event.executor.MethodHandleEventExecutor.execute(MethodHandleEventExecutor.java:37) ~[patched_1.11.2.jar:git-Paper-1078]
at co.aikar.timings.TimedEventExecutor.execute(TimedEventExecutor.java:78) ~[patched_1.11.2.jar:git-Paper-1078]
at org.bukkit.plugin.RegisteredListener.callEvent(RegisteredListener.java:62) ~[patched_1.11.2.jar:git-Paper-1078]
at org.bukkit.plugin.SimplePluginManager.fireEvent(SimplePluginManager.java:517) [patched_1.11.2.jar:git-Paper-1078]
at org.bukkit.plugin.SimplePluginManager.callEvent(SimplePluginManager.java:502) [patched_1.11.2.jar:git-Paper-1078]
at org.bukkit.craftbukkit.v1_11_R1.event.CraftEventFactory.callEntityDeathEvent(CraftEventFactory.java:395) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityLiving.die(EntityLiving.java:1100) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityZombie.die(EntityZombie.java:429) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityLiving.damageEntity(EntityLiving.java:954) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityMonster.damageEntity(EntityMonster.java:44) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityZombie.damageEntity(EntityZombie.java:163) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityPigZombie.damageEntity(SourceFile:148) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityLiving.e(EntityLiving.java:1195) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.Block.fallOn(Block.java:527) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.Entity.a(Entity.java:1046) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityLiving.a(EntityLiving.java:188) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.Entity.move(Entity.java:842) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityLiving.g(EntityLiving.java:1787) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityLiving.n(EntityLiving.java:2081) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityInsentient.n(EntityInsentient.java:502) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityMonster.n(EntityMonster.java:24) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityZombie.n(EntityZombie.java:155) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityLiving.A_(EntityLiving.java:1916) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityInsentient.A_(EntityInsentient.java:245) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.EntityMonster.A_(EntityMonster.java:28) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.World.entityJoinedWorld(World.java:1740) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.World.h(World.java:1715) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.World.tickEntities(World.java:1532) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.WorldServer.tickEntities(WorldServer.java:621) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.MinecraftServer.D(MinecraftServer.java:898) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.DedicatedServer.D(DedicatedServer.java:403) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.MinecraftServer.C(MinecraftServer.java:754) [patched_1.11.2.jar:git-Paper-1078]
at net.minecraft.server.v1_11_R1.MinecraftServer.run(MinecraftServer.java:652) [patched_1.11.2.jar:git-Paper-1078]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_112]
Caused by: java.lang.NullPointerException
at one.lindegaard.MobHunting.PlayerSettingsManager.getPlayerSettings(PlayerSettingsManager.java:37) ~[?:?]
at one.lindegaard.MobHunting.MobHuntingManager.onMobDeath(MobHuntingManager.java:751) ~[?:?]
at com.destroystokyo.paper.event.executor.MethodHandleEventExecutor.execute(MethodHandleEventExecutor.java:35) ~[patched_1.11.2.jar:git-Paper-1078]
... 33 more
The times when this occurs though, whilst the same area is located twice, it's in separate debug sections.
Every crash I've just looked at in the logs, occurs in a similar situation to that 2nd code block above, where one debug section finds the same place twice at the same time. Perhaps that helps narrow it down? As there is a pattern to it.
Please test V4.7.5
I have made some changes where I use the ProtocolLib API, I now send less packets and slower, hoping this will solve the problem on Paper Spigot.
The second bug should be fixed in v4.7.5
Yes, maybe it is another plugin some how. Do you know what this is? de.myzelyam.premiumvanish ? Is it a plugin you have installed?
Hey,
unfortunately it still crashed:
https://paste.ee/p/YybGa
However, your plugin isn't the only one on the current thread. Could that be part of the problem?
For now I'll just keep protocollib integration turned off. Does that affect anything else?
Hey,
I appear to have missed part of the log in that link above, but it was triggered when someone did '/mh checkgrinding' if that makes a difference.
And yeah, It's likely: https://www.spigotmc.org/resources/premiumvanish-stay-hidden-bungee-support.14404/
Is it possible there's some sort of clash between what you both use protocolLib for now? As that plugin wasn't listed there before.
Also, does disabling protocolLib integration do more than just stop the torches? If that's all it does, I don't mind keeping it off as I seem to be the only one so far with this issue...
I appreciate you putting the time in to try figure this out. :)
Hey,
Don't let it bother you too much, the feature only seems to be aesthetic so it's not too big an issue and you've been more than helpful in fixing the issues I've brought to you before.
I updated protocolLib a little while ago, it added "•Added debugging mechanism to print out all packet contents (/packetlog)" do you think it would help? :)
as explained in dmulloy2/ProtocolLib#324 its hard to debug a watchdog timeout as the root cause is often invisible.
@SucceededKiller can you provide a timings report while the flame thingy is running? that would make debugging it way easier
@SucceededKiller could you please try to make a timings report using /timings ? https://www.spigotmc.org/wiki/timings/
Sorry for the late response.
I have a timings report ~10 minutes before a crash, would that work? As I don't know when a crash will occur, getting one close to it might be difficult.
If that is fine, do you perhaps have a way for me to send it to you without my plugins and their version being made public here? :)
@SucceededKiller you need to ask @MiniDigger about an email adress or something
ahhh, only if github had private messages....
You can send the link to [email protected] @SucceededKiller
why did you allocate 25gb of ram? lol
everything about 10gb is insane, 8gb looks like enough for you.
another factor that is hurting your performance is a slow disk.
other than that (and hopper lag ^^) your server seems to be running fine.
I really don't think that MobHunting is the issue here, its just the tipp of the iceberg.
Thank you @MiniDigger, but then what? The server stop only happens when he enable Mobhuntings integration to ProtocolLib! And he can reproduce it.... is there anything I can do to solve this problem?
The RAM is from it running out of memory. :P I set it to a level around where it didn't, but probably need to refine it some more.
The slow disk may actually just be excessive saving. :) I didn't want people to lose too much data when the server crashed, so saving was a bit more regular than usual.
Yeah, hoppers are a pain... Some people can incorporate hundreds of hoppers in a design, or make giant trains of them...
But like Rocologo just said, server runs fine. It's had no issues for...ages, but as soon as that integration gets turned on, the crashes return. Something related to it has to be the cause?
Be careful saving too often to disk. I had a similar error (a server stop) once when on of my users decided to save every 2 second. Which crashed the server... I have implemented a minimum value 5 minutes for MobHunting.
@MiniDigger, could the issue be that i place to many torches too fast, not that i think a play alot of packets?
Or could the error be that i should use runTaskTimerAsynchronously instead of runTaskTimer?
Unfortunately the server still crashed. :( TPS drops more and more, players report chat and block lag, then the server crashes.
With protocollib integration disabled everything works fine.
Honestly, don't worry about it, although I do appreciate the time and effort you are putting in. :) Whilst I'm the only person reporting the issue it's not a big deal, and it works fine with integration disabled and I don't have a problem with that.
Maybe if more people have the problem it'll be something that needs revisiting, but until then I don't mind keeping integration disabled and no-one else seems to be negatively impacted.
If you want to keep testing fixes though then I'll do everything I can to help. :)
One thing that does come to mind though...where did you test it? on the surface?
Chances are, my players trigger it using mob farms, most likely underground/in the nether/buried etc. Could you spawning those torches into certain places cause the problem? Like an area where another block exists?
I have made a small change in my code, hoping that this will fix the timeout error (crash) you get when ProtocolLib is enabled.
Could you please test build 386 or newer?
https://fractal.lindegaard.one:8181/job/MobHunting/