I occasionally have problems with disappearing bees, but today when I logged in, I had hordes of duplicated bees ... over 120, probably (won't have final count until i go through tonight cleaning up the mess).

My setup: each of my bee cells is 2x3x2, with an expanded advanced hive on the 2 wide wall. each had 3 bees, with three different types of bees. I also have some breeding cells, that have no hive available.

In the bee hive ones is where i found all the dupes, so this seems to be tied to the adv. hives.

To give a specific example, I had one cell with one Imperium bee (std) and one Insanium bee (maxxed out stats through dna treats). When I logged in today, I have two Imperium and six! Insanium bees. All the Insanium bees are max statted (Very Productive/Metaturnal/etc).

running on ATM7 0.4.14 (official patreon server), which is currently on productivebees-1.18.2-0.9.0.5

I wonder if the chunks they are in are too data dense to save properly, like filling a chest with written books.

It's weird because it only affected the bees in a 1 chunk radius from where I disconnected.

do those chunks have a lot of hives?

Server rebooted and it deleted all the bees outside the hives, nocturnal bees were duplicated and bees in a room for breeding were ok x_X

would it be possible to get a log from the server both from before the reboot and the immediate run after?

uploading a new version which strips as much data as possible from the bees before they are saved, if there's a data overload in the chunk this will hopefully resolve it
the bees will have to exit and go back in the hive before the data is cleared.

i will add that all of my bees before I parked them were in a one chunk area, roughly 120 bees in hives in that area (spread out vertically).

this could be a data issue then, the chunks simply fail to save because of it

hopefully the new version has reduced the data footprint enough, roughly 80% reduction between 0.9.0.6 and 0.9.0.8

Wonderful, do you know if the server had a crash or rollback prior to this?

they did update not long before i noticed. i don't see any crashes (it reboots automatically every 6 hours, and all server starts are related to that)

no rollbacks either.

do you have any fire near the hives?

chunk they are in is forceloaded, so does continue to run when i am offline, so i'm never a direct witness to when it occurs, i just see the fallout.
no fire anywhere near the hives. closest fire would be ... 40ish? blocks above. (base is all one chunk, so made vertical)

one thing i started to suspect on the disappearing side was due to my cell structure. when exiting a hive with three bees, they'll squish around possibly intersecting or spawning in walls, and will pop in to the adjacent cell. i was starting to wonder if they may suffocate in there, causing the mysterious disappearance issue i was having before. mebbe if they're "in wall" during a server reboot is causing the disappearances &/or duplication? but this one is definitely cloning, given the insanium bee and others... i have a dozen or more rose_gold bees now, even though i only had one in a hive last time i was in (thursday?). Many others, but not all, same situation, from vanilla bees to the highest insanium tiers.

again, no issues with cells that had no hive attached. those all still have the same breeds and number of bees i left them with.

without a way to reproduce it, it's really difficult to fix, but I'll run through all the code and see if I can spot any places it might happen

anything i can do on my end to help narrow down possibilities?

cross your fingers I don't accidentally delete all bees in the next update :D

rofl... will do. i keep spares in std. cases stashed away for the disappearances... has helped out so far. lol

This is an issue for me too, last update deleted all bees that were outside (luckily only a few nocturnal and metaturnal). Yet if it's just a reboot of the server bees duplicate.

@Legorfyr you can reproduce this every time?

There's a new version up, would you be able to check if it stopped with that?

Yep, this happens every time, about a month ago I contacted you about this too, when 80+ hives got wiped out after an update. Since then I'm making it night so almost all bees are inside the hives when the server updates. As soon as ATM7 updates I can let you know if the update works.

Do you know if it’s the bees that are outside or the ones inside that are duped?

Either way, if it’s any and all server restarts it gives me a place to start looking

The ones outside, some bees tend to not go in a hive (grave bees have a tendency to get stuck behind graves xD)
So to give some detailed info:
Update = bees outside disappearing
Reboot = bees outside duplicating

and with ATM servers rebooting every 6 hours .... lol :D 151 cloned bees for me.. more so, as there were updates i didn't check in after. Soon, I shall take over the universe with my clones ... mwahah-oh... that's been tried before, hasn't it? :D

Legorfyr: One question ... this did not appear to affect my breeding bees, which are in cells that have no hive access. So they're 'outside' on updates/reboots, but they don't disappear or clone. Have you noticed the same on your end? That this seems to affect only bees who have an advanced hive home?

Yeah, if I have a breeding couple out, they don't get deleted, it seems to only affect those that indeed have a home (hive)

Server updated to ATM7 0.4.16 with productivebees-1.18.2-0.9.0.6.
All bees outside got wiped again, sadly this time a lot again.

Is there a chance the world is loaded up withou PB installed at some point during the update? This would eliminate all bees outside the hive but blocks would come back once the modded was added back in.

Did you loose the breeding bees that don’t have hives in the update too?

The way the server is updated: all mods get deleted and the new ones put in, configs are overwritten iirc.
I didn't have breeding bees out at the time this happened, so I can't give any info on that.
Also, a bit of a weird visual glitch in the hives, checked some, they were empty, yet when it turned day 1 or 2 bees came out anyway.

Does this visual glitch persist or do the bees show up in the GUI once they get back to the hive?

So, it happened again in productivebees-1.18.2-0.9.0.9.
After a server restart, all bees were duped, except nocturnal ones, a few of those vanished.
It's a different set-up now, bees are underground in a 5 x 5 chunk area, 36 hives/layer. It is chunk loaded.

Just to leave my experiences here. ProductiveBees-1.18.2-0.9.0.9 TNP Limitless 5 Server
Have a series of rooms, 3x3x4, went offline, the chunks were all offline loaded.
There were a few timed server reboots while I was offline, no info on day/night.
Many bee types (lapis, emerald, diamond, etc) all were reduced from 5 bees per room to 1.
One or two (redstone) went from 5 to 0.
Hives were in the center of the walls, so no corner hives, but they were in the walls, not free standing, only the front exposed.
Crowding would only have been an issue at dawn when all 5 bees emerge at once.
The next day, I included a BaBee upgrade in my iron bee hive, and when I returned, there were 10 instead of 5.
If they died, they were replaced at a higher rate.
I tried this with my mystical agriculture bees, which are incapable of being bred, and was able to create many copies of them.
My theories to bee death... a) Minecraft vanilla bee bugs b) crowding c) emerging partially in wall/hive and taking half hearts over an extended period of time d) server restart (chunk unload) not recording bee presence properly and deleting them.
I have no direct evidence of any of these.
As a stopgap measure, when I go offline, I make sure to leave several bees per room with a BaBee upgrade to counteract the deaths and just release the excess bees I get into the world to prevent crowding (cause havoc with the neighbors).
Don't know if any of this helps, it's just my experience with the issues being discusssed here and I hope it does help in some way.

Thank you for your input. It seems like chunk loading is a common thing here.

This doesn't happen with vanilla hives. Unlikely to be crowding as 5 bees are not too much for such a relatively big area. It can't be wall damage, as I've made them invulnerable to that due to the janky position in MC.
So it must be that the chunk is not saving correctly when force loaded or it's saving the entity states and the block entity states at different times.
Do you automatically extract combs from the hives and if so, could you as a test, try to put a hopper under the hives and pull from that hopper instead of directly from the hive? The inventory change of the hopper should mark the chunk for saving and will reveal if the problem is with the hive or the chunk loading.

My setup has a pipe from the Pipez mod pull from the bottom of the hive and send all output to a common chest for all rooms bees are stored in for further processing. Bottles are inserted remotely via a Modular router. Would you like me to put a hopper and then pipe from the hopper? I'm not sure how marking for chunk saving works

.

Will do. Will update tomorrow with results. Will try several variations of bees.

I also mark the chunk for saving whenever a bee leaves or enters the hive, but maybe it's not working well so testing with a vanilla thing is best.

Yes, a hopper between the hive and the pipe so the hoppers pulls from the hive and the pipe from the hopper. The hopper should mark it location as changed when the inventory changes in turn marking the chunk as needing saving.

Quick update: Just had a server reboot.
Daytime
I had 2 hives set up with 5 bees each
Upon returning, hive in chunk without hopper had 1 bee, hive with hopper had 4. didn't check to see how many were outside hive at the time, will do that next reset if I'm still up.

Ok, though I think it doesn't matter, the second test was just bizarre. The hopper bees died out all but 1 (5->1), the non-hopper bees gained 1 (5->6), a superium bee that is non breedable doubled across the hall, an insanium bee died out in another room.

My new theory:
There is a chunk save flag for mob entities that is not getting set when the chunk is marked dirty.

If the physical data of the chunk...blocks, tiles, blockstates, tick count, etc. is being marked as dirty and is being saved, but entities are not, then when a chunk saves, the location of the bee is indeterminate. The chunk has info on which bees are in the hive that is saved at a different time than it's info on bees that are out of the hive. Thus, if a bee left the hive and then it saved, we lose a bee, but if a bee entered the hive then it saved, we gain a bee (regardless of breedability).

This is not based on any knowledge of the code base or how saves work, but an intuitive and logical explanation that fits the behavior of the problem and I think is worth investigating. I know my tests don't produce any logical conclusions if we get sync'd saves of world and entities, it's just chaotic randomness.

Edit: Do the bees have a separate markDirty command?

Entities do not have a markDirty, they are always saved on the chunk if they are not marked for removal.

It could also be the other way around, that the hives are not saving properly, the logic would just be the other way around.
If a bee pops out of the hive and then it's saved, the entity is duplicated.
If a bee enters a hive and then saved, the bee is lost.

What's strange is that it supposedly does not happen with vanilla beehives. Since it's so consistently a problem in your setup, could you add some test chambers with vanilla beehives?

I can. I have no clue how vanilla beehives work, but will try to figure it out. As it is now, there's no way I can reliably use bees as a resource because of the risk of a die-out of bees I've worked hours for, but I'd love to help you get this fixed for future packs.

Will let you know how the tests go.

I've managed to reproduce it, but only if I shut down the server the wrong way (force quit). It's not the chunk loading in itself that causes issues, but it's more likely to happen for chunk loaded hives because they are always loaded when a server shuts down.

If you have server logs to look at, can you check that the debug.log has something like this in the end after a restart?

[30Jul2022 00:04:26.760] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: Stopping server
[30Jul2022 00:04:26.761] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: Saving players
[30Jul2022 00:04:26.762] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: Saving worlds
[30Jul2022 00:04:28.917] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: Saving chunks for level 'ServerLevel[world]'/minecraft:overworld
[30Jul2022 00:04:33.561] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: Saving chunks for level 'ServerLevel[world]'/minecraft:the_end
[30Jul2022 00:04:33.565] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: Saving chunks for level 'ServerLevel[world]'/minecraft:the_nether
[30Jul2022 00:04:33.568] [Server thread/DEBUG] [net.minecraftforge.common.ForgeHooks/WP]: Gathering id map for writing to world save world
[30Jul2022 00:04:33.575] [Server thread/DEBUG] [net.minecraftforge.common.ForgeHooks/WP]: ID Map collection complete world
[30Jul2022 00:04:33.591] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: ThreadedAnvilChunkStorage (world): All chunks are saved
[30Jul2022 00:04:33.591] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: ThreadedAnvilChunkStorage (DIM1): All chunks are saved
[30Jul2022 00:04:33.592] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: ThreadedAnvilChunkStorage (DIM-1): All chunks are saved
[30Jul2022 00:04:33.593] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: ThreadedAnvilChunkStorage: All dimensions are saved
[30Jul2022 00:04:33.931] [Server thread/INFO] [FTB Chunks/]: Shutting down map thread

latest.log should have the same minus a few of the lines

Alas, I'm on a server that is hosted by a public group, so I don't have access to the debug files.

Have put in a request for the files, but no response yet. We had a reset with usual random disappearances and dupes. The vanilla hive and oak hive had 3 bees before and 3 bees after, so it's apparently possible to do.

Don't have log yet, but interested development. The bee nest has now got 6 bees instead of 3. That's vanilla stuff.

the server you are playing on keeps shutting down the wrong way then, I would suggest not having the area force loaded

It won't unload. I have turned off all ways I know of force loading and I come back and all my mana pools are full, all bottles used in hives, etc.

Will talk to admin. Sorry if I messed up your testing with my borked server status/admin

Not messed up at all, I set up a server yesterday and did a test myself and confirmed what needed to be confirmed. Why some servers don't shut down properly I don't know. It could be a thread that hangs.
In any case, once a server doesn't shut down properly it's guaranteed to affect force loaded areas.

Ok, for sure our server shuts down wrong. Easy to get around by just going offline before the failed shutdown, I assume the chunks are saved when they are unloaded. The problem is, our FTBChunks is keeping any force-loaded chunks (and one chunk outside that chunk) loaded while offline, so even going offline doesn't resolve the issue. This is what has made testing for me nearly impossible. I have moved bases and set myself up so that my bees are now outside the offline chunkloading space and should save properly when I go offline before resets. I will test and let you know the results, but I expect they will behave properly. (Fingers crossed)

saw someone mention the chunk loading issue and their suggestion was

deleting the "chunks.dat" file before a reboot helped keep chunks unloaded that were supposed to be unloaded

closing this one now

the issue is servers shutting down incorrectly and not saving the chunks properly
chunkloaded bees are more affected because they are always loaded when the server closes incorrectly

to prevent the issue (if the server can't be fixed) use simulated hives