Animania Base

Animania Base

11M Downloads

Peacocks and potentially all other animals that sleep causing crazy high tps lag

Corosauce opened this issue ยท 5 comments

commented

Expected Behavior

Consistent tick time

Actual Behavior

TPS lag spikes, high mean tick time, skipped ticks.

Steps to Reproduce

Have a decent amount of peacocks loaded, in our case there were 24 instances.
Probably requires testing at night

Version of Minecraft, version of Animania, Single Player or Server

animania-1.12.2-1.5.1.1.jar
dedicated server with 4 cores, 3.2ghz

Analysis

A server we have been preparing has been having tps issues for a while, huge intermittent stalls so bad that it would kick people and overall high mean tick time that made no sense.
I decided to put on my debugging pants and go all out.
I left a copy of the server world running in singleplayer for a while with visualvm open, profiling it, found a hot spot in the peacock code:

screenshot_20180830_220236

Being a programmer I naturally investigated.

I might have overlooked some parts, but It looks like, for every peacock, every 5 seconds if they arent sleeping yet, if its night and maybe if some other conditions need to be met, you are iterating 6144 times, each time scanning for entities within a 2 block radius which iterates entity lists in subchunks of chunks etc, very expensive work: https://github.com/capnkirok/animaniamod/blob/1.12/src/main/java/com/animania/common/entities/peacocks/ai/EntityAISleep.java#L195-L223

I'm not sure how many of the peacocks were truely trying to sleep, but a lot of them could be trying to do this nearly at the same time (especially if they are loaded in on the same tick after a restart etc)

This guy noticed this issue too but I dont believe its solved: #83

Steps to Fix

For mod users:

  • Set the config ticksBetweenAIFirings to something crazy high, I set it to 10000 to test, this will probably break other AI though
  • Or just set animalsSleep to false (requires world reload to make sure it takes effect)

After I used either of these above, the mean tick time went from 40ms+ with lots of skipped ticks, to a buttery smooth 7ms, while 1 player in server over spawn

For mod authors:

  • The work needs to be broken up more over more ticks at least, but I would personally recommend a more cache heavy approach to scanning for beds and checking if they are already occupied.

Off the top of my head something like this should work:

  • Cache the locations of found beds where possible, updating and validating the locations incase anything changes.
  • Make the beds be non ticking tile entities that can store either a flag that they are occupied, or a weak reference to the entity that is occupying it
  • When the animal wants to sleep, iterate through this much smaller list of remembered beds and see if any of them isnt occupied.
  • My Koa entities bongo drum finding can be used as an example of the scanning and location caching and revalidating https://github.com/Tropicraft/Tropicraft/blob/1.12.2/src/main/java/net/tropicraft/core/common/entity/passive/EntityKoaBase.java#L1038-L1070 , but theres still the challenge of making sure not to use an already occupied bed.

Hope this helps, you know where to find me if you have questions incase I dont reply here fast enough :D

commented

Thanks Coro for the clear description and suggestions! I've been thinking about caching lately, so thanks for the code sample.

Looks like another round of optimization is in order.

commented

Ok, will update this entry with the latest findings:

  • All animals use the same Sleep algorithm. So that means all animals are affected. However, that doesn't explain why Peacocks tweaked out here, while other animals behaved. There may be something else insidious lurking in my code.
  • Tested with Nallar's Tick Profiler on a Test Server.. using hundreds of Peacocks... saw zero degradation in TPS. Will use Java Visual VM, however, as this is more accurate.
commented

Hmmm yeah I wouldn't be surprised if it required special conditions to make them keep scanning for beds, maybe if I trapped a bunch somewhere, might play around with it when I focus on our WIP server more.

commented

Just to chip in:

On my own server, I've been hunting down TPS loss sources for weeks using TickProfiler.

Occasionally, I'll find an Animania entity, usually a chicken-based one, with a huuuge tick time, an order of magnitude greater than the next-neediest entity. When I teleport to it, I find that it's usually a chick stuck between a wall and a fence, or a chicken that's fallen into a 2-block pit, or a chicken that someone has brought inside into their basement for safekeeping.

To me, this would suggest that being trapped makes the mobs rapidly iterate their pathfinding--they see a water/food/sleep/mating/play object, try to get to it, fail, and immediately cycle another attempt.

commented

I think we have this one solved. We completely changed the search algorithm and only run it once per AI firing (caching it in the process). Gonna do some testing the next coupla days.