Description of the Bug
(Edited by @liach)
We are running your beta 5 version for 1.12 and when the world goes through a save process, in order to have a backup incase of a crash, Railcraft is trying to save all the locations of minecarts in the world, but during the save, something goes wrong, it duplicates the numbers creating ghost carts, and then tries to save again less than 30 seconds later, creating a constant double save loop, but with increasing file size and cart value, up until it goes over 20 GB of file size, and overpowers the ram on the server, usually every 45 minutes after a full reboot. Its also not overwriting the previous save file, but instead creating a new one, my guess because it thinks there are more minecarts than there actually are. We traced down the issue very carefully, its how we know where that issue is coming from. It usually reaches critical mass around 2,000 carts just before the server completely crashes and requires a reboot of the minecraft server.

(Original content below)

Attempting to disable anything in order to reduce or remove the constant saving issue of Ghost Mine Carts just sets up the client to constantly crash.
To Reproduce

1. Place block/cart '...' Go into the mod config settings and disable anything at all. I tested each option.
2. Click on '....'
3. See '...'

Expected behavior

It completely crashes Minecraft halfway through load up
Screenshots & Video

https://pastebin.com/UXR9n0S5
Logs & Environment

• Full log:

• Crash report (If available):

https://pastebin.com/UXR9n0S5
Additional context

Duplicate of #1717

constant saving issue of Ghost Mine Carts

What? Care to elaborate?

Probably means the constantly increasing number of trains. #1668

We are running your beta 5 version for 1.12 and when the world goes through a save process, in order to have a backup incase of a crash, Railcraft is trying to save all the locations of minecarts in the world, but during the save, something goes wrong, it duplicates the numbers creating ghost carts, and then tries to save again less than 30 seconds later, creating a constant double save loop, but with increasing file size and cart value, up until it goes over 20 GB of file size, and overpowers the ram on the server, usually every 45 minutes after a full reboot. Its also not overwriting the previous save file, but instead creating a new one, my guess because it thinks there are more minecarts than there actually are. We traced down the issue very carefully, its how we know where that issue is coming from. It usually reaches critical mass around 2,000 carts just before the server completely crashes and requires a reboot of the minecraft server

@Solverbolt

Mind you share the info of your server, e.g. mod list, whether you have bukkit, etc.

Sure, we are playing on DFC Server group, FTB Ultimate Reloaded. Just came out not too long ago.

here is the link for the mods loaded in.

https://www.feed-the-beast.com/projects/ftb-ultimate-reloaded/relations/dependencies?page=5

Hmm.

Just FYI, disabling the minecarts won't stop this. It applies to Train objects, which are created for everything that extents EntityMinecart.

Disabling the Train module.... might... but I doubt it, its highly integrated in a lot of code. We could probably disable the serialization when the Train module is disabled without many side effects.

yeah, most of use are not using railcraft as much as we could, but the way the modpack creator set this pack up, we need railcraft for steel, as IE was pretty nerfed from the gate. All in all, i have enjoyed your mod for a long time now, especially with colorized tanks filled to the brim with every liquid imaginable. was always entertaining to me to see how many i could build before a server owner came by and asked if i was insane.

Can you try setting forge dormant chunk cache size to 0 and see? I wonder if that affects. Railcraft had some issues with forge dormant chunks before.

I will see what I can do.

You can try running railcraft in a pure game and see if the issue happens. I am running a regular world with 13 trains in total and have not seen that number growing (my environment has only railcraft and has set forge dormant chunk cache to 0)

okay, might be a small conflict with the world save that is set, but it seems like a very odd conflict to start duplicating at an exponential rate like that

I poked this a fair bit back around beta-3 trying to get it to break, we did have some bugs back then. But I fixed all the ones I found.

(Forge was already set to 0 Dormant)

Admin for this server...

https://pastebin.com/3LLneU0s

less than a minute and 4k trains..

@Solverbolt @TheLittleCube Can you upload your railcraft.trains.dat here or upload it somewhere and link here?

It should be in the data folder in your world save directory, in the same folder as villages.dat.

With your data, I can actually analyze if there is still duplications, etc.

https://www.dropbox.com/s/wn0y3yj2tynycrj/railcraft.trains.dat?dl=0

thelittlecube, is your map fairly explored? If users explore the map often, many mineshafts containing minecarts will be generated, which would create new trains.

As I see in your data file, there is no duplicate minecart in these trains.

Still, there is a possibility that trains are not removed after a minecart is dead. We'll see.

We are about a week old. However there is no way thousands of trains are being gen'd. I have the LagClear running to remove everything that isn't player based. None of the players that I administer have come forward to even using them.

Okay, the other admin has stated that we already used a mod to remove all other dims, than the ones we wanted set, in this case, we only have 4 in total, Overworld, Nether, End and an Aroma Mining Dimension.

He also states that we set dormant chunk cache size to 0 to no effect, still saving over 2100 trains. He apparently adjusted the ram size, so the numbers are now going higher.

@Solverbolt I still need the railcraft.trains.dat from you. Please!

A possibility is that you have tons of minecarts loaded to the game before (from mineshafts)

My worlds regularly go over 2000 trains after generating just a 1000x1000 block area, and I'm the only player. There's a lot of things in mod's worldgen that uses entityminecart. If RC is saving 2000 trains, it's not necessarily wrong, but there's clearly something going on elsewhere. I don't even notice a lagspike when my server is saving 5 or 6000 trains. To summarize, I'm not saying something isn't wrong, but a week old server even with only one player probably has generated enough chunks to hit 2k "trains".

The issue remains. A few hours ago I rebooted the server. And only one player logged in and instantly the server saved 3k trains. No new chunks being generated. Dude was in his base for a split second before railcraft ran. With the entity/item removed every 15 mins still makes no sense. However I'm just wondering where these "trains" are being placed. Cause looking around everything is removed; save for a few blacklisted things. (passives and boss mobs) Hoping for a better understanding possibly a fix. Rather than just allocating more memory for the server.

We haven't released an update yet. Not until ftb ultimate is updated could we confirm whether the bug is fixed.

My worlds regularly go over 2000 trains after generating just a 1000x1000 block area, and I'm the only player. There's a lot of things in mod's worldgen that uses entityminecart. If RC is saving 2000 trains, it's not necessarily wrong, but there's clearly something going on elsewhere. I don't even notice a lagspike when my server is saving 5 or 6000 trains. To summarize, I'm not saying something isn't wrong, but a week old server even with only one player probably has generated enough chunks to hit 2k "trains".

Hmm, the trains should be fine now. Mineshafts are indeed that common.

now we just need to make sure that this is actually fixed.

Hmm, no other reports. Will close for now; report here if it happens again in the next release, make a new issue with better information.

I would love to update FTB Ultimate reloaded with this fix, as loads of reports are flying in with the train saving server lag. (We are also experiencing it in our testing worlds) But I have beta 5 in the pack since mid Feb, and that is the latest version I see on CF, released on Feb 12th. So I really cant test this in the pack if the fix is not publicly available. And if it is and I missed it, I apologize in advance.

Currently, our solution was to remove the log message, as we could find no evidence, or at least no one could provide us any, of there actually being any lag caused by the train saving.

And this was despite several profiler reports and sample dat files being sent us.

For all we could tell, it appears to simply be a case of "the nail that sticks up, gets hammered down" or "the squeaky wheel gets the grease".

If you think it is too slow, my best suggestion is to add a performance mod or use an optimized minecraft server implementation that writes world saved data to actual files asynchrously, especially given that Java is a language slow at I/O and the data has already been extracted from different formats to an compound named binary tag. An implementation can be seen at https://github.com/Akarin-project/Akarin/blob/7487ac4edf5bd518b924ab4891feb67d6af0e1a2/sources/src/main/java/io/akarin/server/mixin/optimization/MixinPersistentCollection.java#L33-L45. Though this implementation is paper-based, it is portable to the forge environment.

For @TheLittleCube, can you post a spark profile or aikar timings profile for us to analyze? I can help you find out the actual source of lag, no matter if it is railcraft related.

The only temp fix I have been able to use; is by deleting the railcraft.dat file from the server. Again its only temp. My pastebin already shows that it took no time to bog down the server.

https://pastebin.com/3LLneU0s

Its so bad that we are just considering just removing it from the modpack altogether just so we can play.

edit: I haven't seen this happen anywhere else with Railcraft in any other modpacks.

Saving 2k trains is not slowing your game down. Its just not. The train saving takes about a 1/10th of a percent of the whole autosave operation. We have the graphs to prove it.

2k trains (aka 2k minecarts), is not an unusual number of minecarts to be created due to world gen.

And while doing the original testing, I've had instances while testing where the train count reached near 2mil due to a bug (which was fixed ages ago). I noticed zero slowdowns despite this.

And if you actually have a several megabyte or larger train dat file, send it to us.

And if you still believe its an issue, run a profiler on it and send us that.

There just is no evidence, other than anecdotal, that Railcraft's train saving is at fault. And we looked everywhere.

Can confirm that I have a world with the same amount of trains... it will grow based on how much area space you have explored on the map). This map has an area of approximately 10K blocks by about 1K-2K blocks. This doesn't lag my world any bit. Use Lag Goggles mod. You can profile FPS and World Ticks separately. It will show you rolled up information of what is actually causing lag on the server. At no point has any railcraft block or entity figured significant in the results. I'm sure there are other profiling mods as well.

https://minecraft.curseforge.com/projects/laggoggles

For that matter, look at the timing of the brief server log provided.

The first:

[20:16:32] [railcraft]: Saving 2,232 Trains...
[20:16:36] [minecraft/MinecraftServer]: Running 3557ms behind, skipping 71 tick(s)

Can -sorta- be correlated to the train save, depending if the save happens exactly the second or not.

But the second:

[20:17:18] [railcraft]: Saving 2,234 Trains...
[20:17:19] minecraft/MinecraftServer]: Running 9269ms behind, skipping 185 tick(s)

Pretty clearly suggests that the server has already started lagging a full 8s before the "saving" message.

Even if one argues that 'saving' appears at the end of the update, and not the start, then the first example falls apart.

FTB Ultimate v1.4.0 is releasing today with still RC Beta 5 as that is the latest I see on curse forge at this time. So not sure how we are supposed to test it in the pack when its not avail to us.

We haven't released an update yet. Not until ftb ultimate is updated could we confirm whether the bug is fixed.

There is no bug to test. We just went over this several posts up.

Just tested something that had been nagging me. Placed a 6 track loop in my base, with 2 minecarts with chests. The carts broke from our Lag-B-Gone that goes every 15 minutes, items dropped on the ground, but the carts are still showing as being there according to the server. But the carts are not actually there. Something is conflicting between Railcraft and the Lag B Gone items and entity clearing script

that was exactly the bug i have fixed in release candidate 1, but it's not public for failing to meet the patreon goal.

@nan0bug00 @Notawallplan
Bug fix is not being held ransom. The bug fix is implemented in a current beta, with a public release pending their fixing some other bugs. The only thing that patreon is preventing is an incomplete, buggy beta being available to the public.

That would fly IF there were no plans to release otherwise. But they were hoping to have a public release with further bug fixes as well in about a week's time, and this beta has the flux Transformer in the middle of being migrated to the new logic

The patron, due to them donating, are allowed early access to builds. Notice THE EARLY, not exclusive.

These are builds that are not ready to be released just yet do to them not being ready for playing on just yet.

The patron goal metioned IIRC is that all early builds be accessible. This was the same issue when patron had an early testing build for 1.12.2 but the latest on curse was not for 1.12.2.

Their is no issue with BETA builds (which this still is, just look at the code if you believe otherwise) being patron only. Personally I am fine with other people paying to bug test Railcraft if this means that I get a better free version and if this within repo is not flooded with bugs that those beta testers already reported.

Just my two sense. Also I have a mod pack with railcraft, generated 1000 chuncks in all directions and have no lag from trains.

@CovertJaguar , @liach
Can you please stop scarring your reputations over and over again? Not even EA will hold a bugfix ransom until players pay for it.

There is no bufix ransom. Patreon is for release of pre-release builds. If the goal hadn't been previously met, you wouldn't even have the version you are playing with. You are pretty rich to get on the internet and accuse a mod-maker(s) of being worst then the worst game developers (EA) for sacrificing their PERSONAL TIME to consistently rewrite a mod from scratch for each version of Minecraft for near nothing in compensation. Seriously, I guess people will say anything on the internet, but this is INCREDIBLY RICH.

Guys, if you want to talk, go back to https://twitter.com/tfox83/status/1106608228404592640?s=21

A few more things I forgot to tell:
The patron goal only affects betas. It has nothing to do with releases. (Also explained in #1724 (comment))

#1724 (comment)
The bugfix is available. Read our freaking wiki before saying "it's private"!

#1724 (comment) Is totally off-topic.

For #1724 (comment), we do not withhold serious bugs that can cause immediate crashes or item duplication. However, this is not big at all: 1. you can fix it without changing code but just deleting railcraft.trains.dat every few days; 2. This does not eat memory at all, compared to resource leaks like ForestryMC/ForestryMC#2384 in which people don't feel anything because the logger did not report! As a result, don't take logs as granted.

I will unlock this issue, but any comments not related to how to fix this bug will be marked as off-topic.

Oh great. Let me just get out my credit card and donate so you can release this fix and hold others for ransom later. Yeah right.

I'm the last person who would ever fault someone for wanting to get paid for their work, but pay-for-bugfixes is a shit business model.

Again, there is no bug here to fix. The whole of this issue is an unfortunate conflation of a bit of server console spam and unrelated can't-keep-up messages. Nobody is missing out on a critical server-saving bug fix.

RC1 (the wordplay there amuses me) will be released unto the world in due course.

@CovertJaguar , @liach
Can you please stop scarring your reputations over and over again? Not even EA will hold a bugfix ransom until players pay for it.