[Feature Request] Automatic profiling on lag spike
Darkere opened this issue ยท 6 comments
We are currently being plagued by some specific bug which randomly causes the server to freeze until the watchdog causes it to crash.
As it's crashing it's not possible to generate a spark report about it making it very difficult to figure out what's causing it.
My request would be a feature where an automatic profiling can start if a tick takes longer than.. 5 min or so.
Did you manage to profile your server's process? I'm running into the same issue.
"Plague" is a good definition for this, the server crashes once or twice a day, and there's no way I can profile it properly with spark.
Also if you'd send your watchdog for comparison that would be interesting too.
No. I've not found a way. However, I'm running into a similar issue, this time with a memory leak where I would need a heapdump under a certain condition. So I'm considering adding this feature into my own mod instead.
FYI, I found, and managed to get to work a profiler that works for my case (Ubuntu terminal):
https://askubuntu.com/questions/1248311/running-jvmtop-profile-the-java-home-environment-variable/1250901#1250901
May not be what you need though.
On large spike watchdog reports the lag spike in console, showing the stack trace, where you can exactly see what caused that spike. Also in crash there will be crash log with all dumped threads and their stack traces at the moment. So you need to look at the Main Thread.
Main thread would be called Server Thread. Unfortunately, the stack traces that are produced there have proven completely useless many many times in the past.
Due to the instant cut off nature it often only shows parts of the problem and doesn't point towards the actual problem.
Also while there is potential that it points to the right direction, convincing mod devs that the problem is on their side (and the game didn't just coincidentally run their code) is difficult.
