CC: Tweaked

CC: Tweaked

57M Downloads

Buffer filesystem changes and sync on world save

SquidDev opened this issue · 4 comments

commented

Currently, a computer gets a direct view of the host computer's filesystem, meaning that when a computer changes a file, it is immediately written to disk. This is fine most of the time (and easy to implement!) - however, in the event of crashes, it's possible for computers to write files to disk, but the chunk they're in rolled-back. This is especially irritating when trying to track turtle positions by saving to a file.

The solution1 here is to buffer file changes, effectively maintaining a virtual file system until the owning chunk2 is saved, at which point we can sync the files back to disk.

There are several other motivations for this:

  • Using a VFS should make it much easier to implement read-write file-handles. As #1214 (comment) discusses, with the current implementation it's quite hard to accurately track file sizes.
  • This is very much a prerequisite for persistence (#535), and should give us a better idea of how we want to approach that issue.

However, there are some concerns I have here - it's a common pattern for people to touch the computer's file system directly, viewing and editing files in an external text editor. This is pretty incompatible with a buffered file system (changes in one will not be immediately visible in the other), and I'm not sure how to reconcile that3.

It may be sufficient to handle this one-way, picking up changes made externally in-game. This at least allows editing files out-of-game, but will not allow (for instance), tailing log files that a computer writes.

Not sure. I'm definitely a while away from wanting to implement this, so any thoughts welcome!

Footnotes

  1. Well, an improvement. There's always going to be crashes which leave things in an entirely corrupted state, but we can at least do better than the status quo.

  2. Or owning entity for pocket computers.

  3. We'll probably want a config option for this, but really we want a something which works Well Enough straight away.

commented

Probably breaks the 4th wall too much but what about a magic flush on files? So the normal fileHandle.flush() would flush to the VFS but another method would flush to the real file system too - thus allowing log files to be tailed outside and position information (in another file) to be in sync with the chunk.

commented

I guess there are also some questions here about how to handle case-sensitivity of filesystems. We need to make sure the in-memory representation mirrors what actually happens on the filesystem.

This would be so much easier if we could just store the filesystem as a single file (like a virtual disk image), but that's probably not very useful for people :).

commented

I guess there are also some questions here about how to handle case-sensitivity of filesystems. We need to make sure the in-memory representation mirrors what actually happens on the filesystem.

I believe that Windows 10 and later allows users to set a folder as case-sensitive, I remember something like this being introduced with WSL 2. I have no idea if it's possible to do that within Java and it might end up causing compatibility issues for older Windows systems - having said that, Microsoft has dropped support for all but the latest Windows 10 release so maybe we shouldn't worry about compatibility with Windows versions that old.

commented

Oh, interesting! I thought this was a file-system wide option - hadn't realised this was per-directory. Unfortunately it looks like this requires WSL to be enabled, so not sure if it can be relied on.