Applied Energistics 2

Applied Energistics 2

137M Downloads

Server side TPS reduction through scale.

Darkpye opened this issue ยท 8 comments

commented

This is not a bug report of any kind but I am trying to find out some information with regard to an issue we are experiencing on our server with regard to TPS performance being impeded by the extensive nature of our ME network.

We are running a RR 3.2.7.2-RC server (218 mods)
4GHz Core i7; 16GB Ram
Applied Energistics2-RV2-Stable-10
Running on Minecraft 1.7.10

We have been working on this world for about 18 months, and have a fairly extensively network setup with literally thousands of devices and unfortunately due to the overall scale of the in game system the server is just not keeping up.

To this end we are looking into how we can modify/edit the system to make it less server intensive. Just to clarify the network doesn't simply run thousands of devices constantly, most of the machines hooked up are only used on demand most of the network devices are idling 99.9% of the time. Only Systems that perform essential functions (such as power supply) that run all the time.

So my first question: do AE devices (import/export/storage buses) that are not importing items use up any server processing power?
Do import buses that have nothing to pull into the network cause any strain? Or export buses trying to export into a full inventory?
Do sub-networks put more strain on the server (we have a very modular system with many facilities on sub-networks communicating with the central auto crafting network via interfaces)?
What kind of effect to Crafting computers/Molecular Assemblers that are idle have?
Do P2P tunnels add any strain. (once already programmed)

I am trying to decide whether using toggle buses, to switch off large parts of the network will have any effect on server performance. Part of the problem being is I am not aware of a way in which have the network turn on an entire facility when a user requests a crafting order that would need that facility. Otherwise I would probably enforce this as a matter of protocol.

Lastly perhaps someone may have some insight as to why this would happen. We have a storage sub-network that holds all our storage, it contains ME drives and some Storage bus's connected to Deep Storage units for items such as cobblestone/dirt. When we turn off the Storage sub-network the servers TPS immediately returns to maximum TPS,

  • Minecraft Version: 1.7.10
  • AE2 Version: RV2-Stable
  • Forge Version: 1.7.10-10.13.4.1448-1.7.10-universal
commented

try removing only the DSUs. they tend to cause issue.
That is all I can tell. Maybe exchange them for barrels as a test?
But stuffing a whole storage network behind a single interface caused a lot of issues in the past for me, can't you try to integrate it into the main network somehow ? (like a dedicated set of P2Ps over a single dense wire)

commented

Thanks will give that a try, we originally had it on the main network but moved it to a sub-network for if whatever reason the ME-Network goes down, we can simply power up a small part of the network to access the drives. Most of our power is provided by steam turbines which means they can take a while to get up to speed while we wait for boilers to reach high enough temperatures. Long story short, it having a small power overhead has been useful. But if having it on the main network improves performance I am sure we can find an alternative solution to the power problem.

Edit: The other reason was so that it was easier to expand it as we wouldn't have to pull channels from the main controller or pull more cables to add a new drive or something. This was inline with our macro design philosophy. (Which we are currently reviewing due to the current issues.)

commented

togglebus and inverted toggle bus side by side (controlledby the same RS signal) with one cable leading to alternative controller with terminal and alternative power :P

commented

In short, guessing does not help. If you cannot profile your server, it is impossible to really point at any issue.

I honestly have never heard of DSU being an issue, they certainly should behave better normal chest or things like storage drawers. No idea where that rumour started, but it seems to be easier to spread bullshit instead of actually providing provable information. Maybe because that is actually the hard part.

The general tips are usually

  • Chunkload everything with AE2. Constantly reloading is way more taxing than keeping them loaded. Anything which forces a more aggressive unloading is probably causes more issue than it tries to solve
  • According to #2495 ae2stuff machines can be really bad in some cases.
  • Be very careful with extracells. Like each fluid storage bus can force the network to rescan every single inventory attached to it. Multiple times per tick.
commented

Thanks yueh, I can confirm that we do use Extra cells, so I can check that, perhaps separating the fluid storage out of the main storage network will offer some help. When you say it rescans each inventory does this apply to chained sub-networks i.e. if I have a fluid storage sub-network and an item storage sub-network each attached to the main network, and I have a separate facility which requires access to the fluid storage and I place a fluid storage bus onto an interface on the main network, will that scan, basically be scanning every inventory as well or will it see the 2 storage buses on the main network only and stop there?

We are using a server profile mod to assist us in trying to figure out where the biggest issues are, it's pointed to the MFR harvesters and Energy cubes from Mekanism being the biggest individual culprits that are hogging server resources, which we are finding ways to address, unfortunately AE2 is just not featuring as problem in the profile. So I am not blaming AE2 as the problemetic so much as we just have too much going on.

I am fairly certain that the entire network is chunk loaded, I will double check it however to make sure that that is not causing any problems.

AE2 Stuff is not included on mod pack, I don't believe it adds any functionality to AE2.

To give you and example of one of our facilities: The inscriber plant, is a high speed processor factory, it runs 14 inscribers in parallel, per operation so 14 for each press and then 14 for each of the processors.
so that's 98 inscribers. The crafting interface stores the items in the inscribes when crafting is requested, and for the processors the secondary inputs are exported into to inscribers, the pattern only calls for items that come from the presses. When this facility is idle, will it be making any requests per tick? There are about 100 storage buses, about 100 export buses. and about 100 import buses so this alone has about 400 odd devices in it.

commented

I honestly have never heard of DSU being an issue, they certainly should behave better normal chest or things like storage drawers. No idea where that rumour started, but it seems to be easier to spread bullshit instead of actually providing provable information. Maybe because that is actually the hard part.

locked DSUs were a reason for items getting deleted randomly , that is a fact and it is unclear until today of the exact reason.

More important, it is proven that ME system starts acting funky with only EXU installed and DSUs beeing target of fast item interaction(somebody was feeding an ME from an highspeed ender quarry), hangig up assemblers and auto crafting when using DSUs as storage, i can chain you the issue numbers if really required

commented

Deleting items does not mean a performance problem. But stating DSU are a problem, when someone ask about performance problems, it is plain wrong.

Similar it is wrong that they are an issue in combination with ender quarries. It has nothing do due with DSus, but just any inventory out there. With DSU on the better side compare to simple chests. It is mostly about how many distinct itemstacks are entering the network. Pushing 64x 1 cobble into the system is also 64 times worse compared to 1x 64 cobble.

@Darkpye pretty much everything in AE2 has a dynamic tickrate. Import buses for example can scale between ticking once ever 5 ticks up to once every 40. In most case they even go to full sleep. In case of import buses this should happen some time after the inventory is actually empty and only wake once the inventory indicates something changed (or chunkloading etc). Storage buses are pretty much passive in terms of subnetworks and act similar to import buses with normal inventories.

commented

Again thanks for the feedback.

This information is very helpful, and reinforces my initial expectations as to how I expected the ME network to function. I will only be able to continue trouble shooting the problem again this evening. But the insight into extra cells is very helpful and I am hopeful that may fix the problem. If it is a fluid storage bus issue, it is probably is the last place I would have looked, because we really don't store large amounts of fluid in the system. It's primary function hold lava and for the transfer of sludge from various places to a central sludge boiler. Seriously though, I have had endless issues with the Fluid bus, it really lacks the polish of AE2.

On a side note, I am happy to check if the DSU's have any effect on the system, would require to cut 1 ME cable to check so it's not a big deal from a trouble shooting point of view. I'll give feedback.

Again thanks for the information I appreciate all of your time.