Better Handling on Disk Full IOException

Question

Better Handling on Disk Full IOException

Zottelchen opened this issue 4 years ago · 1 comments

Zottelchen commented 4 years ago

Is your feature request related to a problem? Please describe.
I've tried to render a giant map (uncompressed because my webhost only takes uncompressed files). It filled the disk (220 GB at ~50%). Then it only threw errors. All progress is lost, I think.

Individual file errors

[ERROR] Failed to save hires model: web/data/drehmal/hires/x-1/5/1/z-8/0.json
java.io.IOException: No such file or directory
      at java.base/java.io.UnixFileSystem.createFileExclusively(Native Method)
      at java.base/java.io.File.createNewFile(File.java:1027)
      at de.bluecolored.bluemap.core.render.hires.HiresModelManager.save(HiresModelManager.java:95)
      at de.bluecolored.bluemap.core.render.hires.HiresModelManager.save(HiresModelManager.java:86)
      at de.bluecolored.bluemap.core.render.hires.HiresModelManager.render(HiresModelManager.java:80)
      at de.bluecolored.bluemap.core.render.TileRenderer.render(TileRenderer.java:49)
      at de.bluecolored.bluemap.common.MapType.renderTile(MapType.java:70)
      at de.bluecolored.bluemap.common.RenderTicket.render(RenderTicket.java:42)
      at de.bluecolored.bluemap.common.RenderManager.renderThread(RenderManager.java:164)
      at java.base/java.lang.Thread.run(Thread.java:832)

And even more fatally it also failed to save the render-state.

Render-state file error

^C[INFO] Stopping render ...
[INFO] Saving tiles ...
[INFO] Saving render-state ...
[ERROR] Failed to save render-state!
java.io.IOException: No space left on device
        at java.base/java.io.FileOutputStream.writeBytes(Native Method)
        at java.base/java.io.FileOutputStream.write(FileOutputStream.java:347)
        at java.base/java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:252)
        at java.base/java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:210)
        at java.base/java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:145)
        at java.base/java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:187)
        at java.base/java.io.DataOutputStream.writeInt(DataOutputStream.java:196)
        at de.bluecolored.bluemap.common.RenderTask.write(RenderTask.java:180)
        at de.bluecolored.bluemap.common.RenderManager.writeState(RenderManager.java:241)
        at de.bluecolored.bluemap.cli.BlueMapCLI.lambda$renderMaps$0(BlueMapCLI.java:142)
        at java.base/java.lang.Thread.run(Thread.java:832)
        Suppressed: java.io.IOException: No space left on device
                at java.base/java.io.FileOutputStream.writeBytes(Native Method)
                at java.base/java.io.FileOutputStream.write(FileOutputStream.java:347)
                at java.base/java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:168)
                at java.base/java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:237)
                at java.base/java.io.FilterOutputStream.close(FilterOutputStream.java:188)
                at de.bluecolored.bluemap.cli.BlueMapCLI.lambda$renderMaps$0(BlueMapCLI.java:145)
                ... 1 more
        Suppressed: java.io.IOException: No space left on device
                at java.base/java.io.FileOutputStream.writeBytes(Native Method)
                at java.base/java.io.FileOutputStream.write(FileOutputStream.java:347)
                at java.base/java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:168)
                at java.base/java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:237)
                at de.bluecolored.bluemap.cli.BlueMapCLI.lambda$renderMaps$0(BlueMapCLI.java:145)
                ... 1 more

Describe the solution you'd like
Check every now and then if there is still enough space and if not cleanly exit and save the render-state.
(Also is there a way to recover from what I already have?)

TBlueF · Answer 1 · 2022-10-14T13:22:56.000Z

After thinking about this and the ways this could be implemented i think i don't want to implement it at all.

BlueMap is now working based on an abstract Storages, which can be implemented based on a File-System or something else like an SQL Database.

The problems with this now are:

Each map can be rendered onto any of those storages
- one storage reporting to be "full" would stop all maps from being generated?
- Or just the one that's on that storage?
Not all storages support checking available space
BlueMap would need to make assumptions on how much space it needs to be fine for the next working-step

So this would already be very confusing to display and debug.

Additionally, this should just never happen. You always need to have enough disk-space left. Minecraft itself doesn't check this either (iirc) and just dies or corrupts the world if the disk is running full.

If a corrupted map is the worst that happens if the disk-space is full then: Yay! All that data is recoverable by just rendering again :)

My conclusion here right now is, that it's too complex and difficult to implement properly for the small improvement it will be.

Closing this issue for now, but i am open for any further discussion :)

Share to