Project Red - Expansion

Project Red - Expansion

31M Downloads

[Fabrication] Complicated IC Blueprints slow down the game in various ways

kitlith opened this issue · 9 comments

commented

I can make a video showing these if necessary, but for now I'll just describe them in text. All of these cases I noticed while using the following blueprint in a single player superflat world based off of the redstone ready preset.

Blueprint (Click to Expand)

8bit_dualreadx4ic

  • Looking a single copy of the blueprint drops FPS from 60 fps to 30fps. This includes:
    • Looking at the blueprint in my hand
    • Looking at an item frame containing the blueprint (compunded by #1365)
      • Looking at multiple item frames containing this blueprint compounds the effect as well.
    • Looking at the blueprint in the IC Workbench
  • Some operations take a long time to complete. This includes:
    • Placing an IC chip based off of this blueprint freezes my client for a few seconds.
      • Whatever happens is happening before "can I actually be placed here" in some cases, because it'll still freeze when it can't actually be placed -- for instance, on a piece of wire.
      • Copying these ICs using World Edit freezes the server for a while, and places them one at a time.
    • Opening the IC workbench when this blueprint is on it leaves the client responsive for a few seconds before the UI actually comes up.
    • All kinds of changes may take a a few seconds before they show up in the IC Workbench
commented

Now looking at how to save stuff, am I correct in saying that SEIntegratedCircuit essentially holds the compiled version of the circuit, which is a list of registers, a list of gates, and a list of dependencies for each register?

There are two main approaches for saving (I think):

  • Save the compiled circuit to global state.
    • Load once, refer to it multiple times.
    • How do you clean up old compiled ICs?
    • Someone could bugfix a ton of tiles at once by updating the global store.
  • Save the compiled circuit to every single item/tile.
    • Reload every time, save bloat?
    • don't have to worry about cleaning up old stuff -- if there are no items/tiles of it, it's not there.

I wouldn't worry too much about how much time it takes to save/load this stuff (at least at first), as it's probably far less than recompiling it every time atm.

EDIT: I'm overthinking it. You're already doing the latter with the tilemaps.

commented

It turns out the usage of distinct on a Seq in WireNet.mapChannelForPoints->iterate was the largest cause of issues. Switching points over to a set instead of a seq made it a whole lot faster.

I'm gonna try and see if I can make it a bit better (iirc next largest issue was the use of contains on a seq) and then make a PR. Unfortunately, I'm out and about at this moment, but I should be able to do that in a few days.

EDIT: Now that I'm home I'll just push up what I have and PR, I don't want to change something that might actually be order dependent without discussion. Where's the best place to talk about stuff like this, @MrTJP?

commented

It looks like the information that SEIntegratedCircuit has for the gates is insufficient for saving, because all it keeps around is an ISEGate trait object that only allows access to a method that performs the logic.

Knowledge we need to save:

  • Gate Type
    • This is probably easy to add to ISEGate.
  • Outputs of a gate
    • We can (probably) already reconstruct this from regDependents
    • but do we need the order of the outputs?
  • Inputs of a gate
    • Could maybe add this to ISEGate?
    • Or maybe add a gateDependents to SEIntegratedCircuit?
    • do we need the order of the inputs?
  • Maybe more?
commented

Something else that's unconfirmed, but I think it also slows down world load. I finished putting 64 copies of the IC in my world last night, and now my world takes around 11 minutes to load. I may try to place down 30 copies in another world and see if it takes around 5 minutes to load, see if I can correlate number of ICs to world load time.

commented

Measuring Methodology: taking the difference between the time the world starts loading and the time the single player world finishes loading in the logs.

World Loading Times:

  • 16 ICs: 2:17
  • 32 ICs: 4:18
  • 64 ICs: 8:33

In between each attempt I used WorldEdit to 'stack' the ICs and double them. It turns out that using WE to copy the ICs takes the same amount of time as placing them manually, but the client remains responsive, even though the server is unresponsive until the operation completes, placing one IC at a time. Nothing else in the world changed between tests.

commented

Some thoughts for what we can do about the server-side issues besides optimization: it seems like what's being done is storing the entire blueprint, and then generating the IC data on the fly when one is placed/loaded. I think the best thing we could do for this is to do it once server side, when the ic chip is generated from the blueprint using the IC Printer (on another thread to avoid blocking the main server thread), and then store/load that data directly. Additionally, when in the IC workbench, maybe move a few more things to the client side and/or have a separate "simulation" mode so that it doesn't need to run through everything on each edit.

Of course, I'm not super familiar with the code-base (or with java/scala!), so... thoughts?

commented

I took a bit of time to profile these cases, today.

Everything I listed under "Operations taking a long time to complete" seems to occur on the server side and have roughly the same cause: the entry point seems to vary, but eventually we get to ArrayGateICTile.buildWireNet() (and to a much lesser extent, WireICTile.buildWireNet()) which eventually calls WireNet.mapChannelForPoint(), whose time is taken up in what is labeled as WireNet.iterate$1() (the iterate function internal to mapChannelForPoint). (Perhaps it's taking a bunch of time because of the arrays of null cells I had to use?) The effect of these is rather serious, editing a single tile on this blueprint seems to take around 8 seconds of CPU time on the server.

As a side note, for placing an IC Gate by hand, it seems to be doing the processing on both client side and server side, which explains why the client freezes as well.

FPS drops while rendering the IC Blueprint seem to come from stuff happening inside {GateICTile,InsulatedWireICTile,BundledCableICTile}.renderDynamic() in this case, which may also vary based on the contents of the Blueprint. This is a much less serious issue, as it occurs client-side instead of server-side, but it's still something that would be nice to optimize.

commented

The system definitely needs optimization. Creating the circuit from the map is very cpu intensive. It is recompiled every time. I need to find a way to save it. Serialize it into bytes and recreate it directly instead of going through the algorithm.

As for the in hand rendering, that’s tricky. Because you can’t render it off a compiled circuit data, only from original source map. But since it’s not being compiled there shouldn’t need to be a need to optimize this much.

Workbench optimizations will come over time as the system matures. Lots that can be done but remember you are editing a live environment. It will naturally be cpu intensive for the client and network intensive for the server.

commented

This is now outdated. Refer to the Fabrication v2 discussion: #1616