[1.11.2] Disks Disappearing
dannydjdk opened this issue · 80 comments
Issue description:
We are having issues with Disks disappearing from our drives. It tends to occur when the server crashes for any reason. When the server comes back up, everyone's drives are gone. Crafting patterns and upgrades are also disappearing, though sporadically. We've also been seeing items disappearing in the Solderer. For instance, when placing 16, we'll end up with 4 soldered, and the rest will disappear. Not sure if that's related to the other issue though.
Unfortunately, the problem is quite intermittent and I haven't come up with a way to reliably replicate it. I'm still working on this, but, considering the severity of the problem, I thought I should let you know about it asap.
What happens:
On server crash (or possibly any restart), disks, patterns and upgrades will intermittently disappear.
What you expected to happen:
Of course, we expect disks, patterns and upgrades to persist when server crashes/restarts.
Steps to reproduce:
- Have RS system with disks and crafters with patterns, etc.
- Crash server with some other buggy mod.
- Restart and possibly freak out cuz all your stuff is missing. ;)
Version (Make sure you are on the latest version before reporting):
- Minecraft: 1.11.2
- Forge: 13.20.0.2227
- Refined Storage: 1.4
Custom modpack/mod list: http://dannyandson.com/unreal-tech-modpack/
Does this issue occur on a server? Yes
If a (crash)log is relevant for this issue, link it here:
No crash log to report. Logs not available at the moment.
I apologize for the lack of info at this time. I realize that this will be quite difficult to pinpoint with the information I've provided, but again, considering the severity, I didn't want to delay the report. Let me know if there's any further info I can provide or troubleshooting steps I can take. My time for troubleshooting is limited, but I'll do what I can.
I did know about this, and it should be fixed for 1.4.1. I gave a (serverside) testbuild to a few people and they didn't report any more of this happening. This is due for release today or tomorrow.
In case you're still searching for a solution to this, I've noticed that whatever is causing the glitch seems to be stripping placed blocks of all their NBT data. Not only are the actual "physical" items lost, but any settings applied to the blocks are also lost (so, for example, settings applied to importers and exporters).
Not sure if you knew this already, but it might help.
Is the update being pushed for this or not, I see there is nothing pushed since February 7th, 2017.
We are having the same issues on our server in 1.11.2, nobody noticed until we upgraded to Forge 2232.
Refined storage drives gone again from machines including speed upgrades and filter item list in export card no crash report from client side I just log in after a few hours and its all gone this is second time in the two days after updating this mod that this is happening.
@digghair Yes, that is nothing more than we didn't know yet. I'm still investigating the issue, I don't know what causes it.
Sorry, we assumed it still was an issue based on your comment above, so we've been using external storage. We'll test it over the next few days and let you know what happens.
I still get this issue. I play on version 1.4.1 in All the mods 2 modpack version 0.50.
I have 2-3 independant RS systems on the server. One of them is chunkloaded for getting items out of the quarry and the other two (meanwhile only one other) isn't chunkloaded. The one that isn't chunkloaded always lose everything when I log in the next day and no one was on the server. I tried using storage blocks instead of disks, because i thought blocks in the world maybe don't disappear like the disks in the drive. The storage blocks are still there but the items on it are gone completely. Also upgrades in the wireless transmitter disappeared.
Everything I described happened just in the non-chunkloaded system, it never happened to the chunkloaded one (I don't know if that is important, but as I use more than one rs system and it happend only to that one I thought it would be worth mentioning it)
Can also confirm this is still happening on dannydjdk's server on 1.4.1. Just happened again to us earlier today.
When this happens, are all disks gone throughout the server or is it only limited to a few people?
Usually it's limited to a few people, but it's not always the same people. It seems random (that's always helpful to know, right?). There was at least one instance where it appeared that everyone lost their items (disks, crafting patterns, upgrades and settings) at the same time. It corresponded with an unrelated server crash.
Our server is configured to allow chunk loaders to operate while the player is online and for an hour after disconnection. This seems to support @vikespear's suspicion that chunk unloading and reloading triggers the problem. In our case, it seems like the disappearances were only occurring after a crash or reboot, but it may just be that that's when we happened to notice because of the mass unloading and reloading of all players chunks after a reboot.
No plug-ins; strictly Forge. And, apparently it has occurred in single player for some as well.
The only place where a node is removed is in BlockNode#breakBlock which is only called when the block is broken:
@Override
public void breakBlock(World world, BlockPos pos, IBlockState state) {
super.breakBlock(world, pos, state);
INetworkNodeManager manager = API.instance().getNetworkNodeManager(world.provider.getDimension());
INetworkNode node = manager.getNode(pos);
manager.removeNode(pos, true);
API.instance().markNetworkNodesDirty(world);
if (node.getNetwork() != null) {
node.getNetwork().getNodeGraph().rebuild();
}
}
This is Still Happening as of refinedstorage-1.4.1 Forge 2233 1.11.2 on The New Fronter Server
I have noticed that the data files are saved/overwritten normally when the server shuts down (Figured this out because I have tried copying the data files while server is running and restarting and the files are not the ones I copied over, they get overwritten, of course shutting down the server then copying the files over and starting back up the server would sometimes work and restore the data I had from a backup) could it be that the data is not saving all the way or the save for the data is canceled when a crash is occurring?
We just had a user loose every one of his drives. forge-1.11.2-13.20.0.2251
One of Squared pack.
He says he had just been intereacting with his RS system then was disconnected.
Here is a crash report around that time - http://pastebin.com/WcWAAk5c
Those are the coords of his base.
Still happening. It's happening with my mod pack as well. RS 1.4.1 doesn't seem to matter which version of forge we are using on our server. It has happened to people with and without chunk loading. If they have drives the drives disappear and if they use the storage blocks the inventory disappears. So far the only common occurrence seems to be when we have a server crash, not a scheduled restart or random reboot. And the crashes we have had were not related to RS. We do use Minecolonies but the problem has happened to people with and without a colony. Not sure if any of this info helps.
Still happening here too! People are getting discouraged by having to recreate their pattern every 2 or 3 times a week... We need a fix.
P.S: It happens with /stop too! Not only with crashs.
I cant reproduce this bug! People who want this fixed should provide me with 100% reproduceable steps.
I will also help as much as possible to collect information since this is one of the favorite mods on our server. I can say that I was not able to reproduce the problem by doing forced restarts of the server which is why I think it is related to when a crash occurs. We also do a scheduled nightly reboot on our server and it's never happened after that either. I have set up some storage systems in different places on our server and will try to provide more info when it happens again.
When you try to reproduce it, what mods are you using in your test environment? Are you testing it in a dedicated server vs the one in the build suite as there its a difference?
I tried this on a dedicated server, yes.
Are you simulating a server crash and also forced restarts?
I'm simulating one by throwing an exception.
Is it possible you have too many exception handlers in place that would otherwise allow crash reports to be generated?
No.
Did anyone that provided you with crash reports have this problem WITHOUT the Minecolonies mod? I removed Minecolonies from my modpack because it was interfering with other mods but I have not had this problem since I did that. Even when we updated to 1.4.2 I created another test setup on our server and have basically been using it as a server wide trash can to see what happens. We have only had a couple of crashes so far, but RS is still intact.
@Cntrldfusion. Yes the original author of this Issue. His modpack does not contain any Minecolonies.
As the writer of the new IItemHandler system for Minecolonies, I can tell you that unless you are giving the disks, patterns etc to you Minecolonies Citizen, Minecolonies is not interacting with them in any way.
We have no compatibility interaction with RS (it is not needed, and provided by default thanks to MC Forge)
We donnot interact with your inventory nor with any other player. We only interact with vanilla chests, to make minecolonies feel as vanilla as possible.
I dug a bit into this issue:
The only thing that might cause this from minecolonies side is its Colony Grief protection. However, the grief protection is based on normal events that forge fires (like BlockPlaceEvent, PlayerInteractEvent etc)
To handle the protection, events are cancelled when a player is not allowed to interact in a given region.
@raoulvdberge are you using a fake player anywere? If so it might be that your fakeplayer is blocked by Minecolony. This also caused issues with BC Builders etc, we discussed this issue with LexManos to see how to proceed, and he told us that the implementation for a FakePlayer should return the UUID of the Player who placed it.
If you already are doing this, then interaction with the colony is guaranteed to work.
I'm not sure if they do or not, but I think it's possible since also around the time this issue started they introduced a Warehouse for inventory storage and a Deliveryman NPC to interact with it.
RS stores every node in WorldSavedData. They are indepentent from the Tile Entity, but if you modify the inventory through the Tile Entity, the WorldSavedData instance will be changed as well.
And we are always asking for the fakeplayer thingy since a lot of mods seem to implement it wrong and then do not check for events that might be cancelled, it caused a lot of issues with for example EnderIO.
If you are not using it to place a block or something that might hold the disk data, then we are not interacting with it.
We do not use Minecolonies and we also experience this problem, do not fixate on this mod.
I've had several disks empty themselves in a similar way. The disks are labeled as full and show as full in the disk drive but nothing shows up in the grid.
I haven't been able to reproduce it yet but I'll keep trying. Let's hope I don't replicate it again on my survival server.
I'll definitely give it a shot to replicate it though.
I'm on version 1.5.32 on 1.12.2. This issue definitely isn't fixed on 1.12, sorry that I posted this in (probably) the wrong thread, but I'd rather not open a new issue when one already exists. I'm gonna go reply to the 1.12 one
Today I saw disappearing items and patterns for the first time in my world (MC 10.10.2, Forge 12.18.3.2185, RS 1.2.25) and I could reproduce it:
Lived quite well with the chunk loading ward of Extra Utils 2 for a long time and wanted to change to Chicken Chunks (v2.2.0.52), so that I didn't have to build that many wards. After placing the chunk loader, half of my patterns disappeard and also a whole lot off other items.
Restoring my backup, everything was fine again and I could reproduce this behaviour (place chunk loader -> save/exit world -> reload world -> items/patterns gone). No errors, no crashs.
could there be a correlation to this topic?
Some technical information, in hopes that anyone with a technical background can help out:
RS network nodes (aka machines) are stored in a WorldSavedData (see com.raoulvdberge.refinedstorage.apiimpl.network.node.WorldSavedDataNetworkNode).
On world load event (see com.raoulvdberge.refinedstorage.apiimpl.network.NetworkNodeListener) it reads the world saved data NBT and puts it in a NetworkNodeManager.. next up: in TileNode#getNode, it gets the node from the NetworkNodeManager.
WHAT I THINK THAT IS HAPPENING:
TileNode#getNode is called somehow before the WorldSavedDataNetworkNode is read, causing it to reset the machine data.
I'm not sure about this though, it can be something else too.
WHY CAN'T THIS HAPPEN:
During read, it would just overwrite the earlier set value in the NetworkNodeManager, working around this bug... but I'm not sure.
WEIRD BEHAVIOR THAT CONFUSES ME:
This bug only occurs on some nodes, not on all nodes at once. This means that the corruption isn't world-wide, it's just limited to a few positions. This is possibly related to chunkloading / loading a chunk when a player joins?
WHAT IT ALSO CAN BE:
There is some code that reads out legacy NBT tags from the tile entity, and reads it to the new decoupled NetworkNode (for the 1.10 -> 1.11 conversion).
What could happen is that the NBT tag count doesn't match, and it reads out the TE data again, while it shouldn't, and thus voiding essentially all machine data.
This still doesn't make sense to me, as this cause wouldn't occur randomly like it's described here (during server starts / restarts, but not always).
Ok, I reworked the code a little and it should fix this bug.
Can you guys try this build:
https://drive.google.com/file/d/0B2PrXMobPWvWbUVabHhRcDQzQ1k/view?usp=sharing
(Warning: Clients need this build as well, you can't just use 1.4.4 on the client and this build on the server. The debug build is technically 1.4.5)
There is some debug output. If you're having problems, link the logs to me.
I seem to be catching a bit of an exception with the new build. my client happens to be crashing before title screen with the crash log found here: https://pastebin.com/93E0DFTt
The server is crashing as well. that crash log can be found here: https://pastebin.com/LNNjje9v
Ah, that is the OC integration crashing if OC isn't installed. It's 5am here, I can't fix it now. Can you just install OpenComputers for testing purposes and retry?
Okay, both server and client are up and running, I will be testing to see how this runs over the next few days. Will keep you posted
Ill be doing testing over the next few days and ill get back to you, but its all looking good for now :)
Would it at all be possible to get a version released on curse forge so i can push an update to the pack im using?
alright, no worries, ill have it on a test server for the next couple weeks or so, so ill keep ya posted
Cool. In what time intervals did this issue occur? That way, we can put a deadline and know sure the fix worked.
Make sure to restart and update the server as well at times, to stress test and simulate the cause of this problem.
yeah im going to be doing restart tests for sure, and ill be pushing pack updates during that time as mods update and become available.
but to be honest most of the times we had it happen was when we updated Refined Storage. It didnt happen when i updated to the debug build which is a good sign, but i will be keeping an eye on it for sure
It still has the debug output, see my reasoning here:
1.4.5 will be coming out on Curse in a bit. I decided to push it anyway.
So far its working beautifully. through all the updates and restarts we have had no isssues so far
Any updates @Tithonius?
I'm assuming this bug is fixed! Please still keep me up to date though. I'm closing it for now.
I just lost my drive again. No crash just upgraded forge and some mods. Not eveyone lost drives and mine was being chunkloaded with Chicken Chunks and was in my claimed land. Not sure if that's part of it but wanted to note it.
Edit: Other users who were chunkloaded with chickenchunks in claimed lane were fine so that was probably not it.
What I am doing for now is connecting a ton of storage drawers to a drawer controller connected to the external storage. @kreezxil
I published version 1.4.7 on Curse.
It has a new config option "debugLog" that generates a .log file with diagnostic info for me, to help fix this bug.
If you are experiencing this bug, please enable it on your server.
If the bug happens again, send me the relevant log file.
Just updated my server after having drives disappear for multiple players today, crossing my fingers this solves the issue. Thanks for the hard work!
Seems to be working, so far no new reports of missing drives! I'll post here if this issue comes up again.
I am having a similar issue, but it doesn't just happen when the server restarts (single player world). Playing Skyfactory 3 and the crafting upgrade I'm trying to put in an exporter keeps disappearing about a second after I put it in the slot, as if it's being exported as well (but obviously doesn't go anywhere).
Actually, just found them somewhere and realized they actually were being exported, not sure if this is the same issue anymore.