Heightmap too large for region file
DaMatrix opened this issue ยท 5 comments
What the title says. I mentioned this in the discord server a while back, then remembered about it just now and decided to report it.
Caused by falling down an infinite hole with random blocks generated on all sides, after about 2 minutes of laggy falling the log was filled with these errors, and new cubes didn't save any more.
I'll end this by posting the old chat log about the error.
Thanks!
I would suggest modifying RegionLib to use 0xFFFFFFFF
header data as "stored externally" flag and if exceeds max size - store in external file
Excuse my ignorance, but where specifically? RegionLib appears a huge tangle of misc. int arrays and bytebuffers. I've set SIZE_BITS
in IntPackedSectorMap
to 16 (from 8) because that's where the stack trace was pointing, is that really bad?
the limitation really comes from 32 bit integers...
one 32 bit int needs to store offset into the region, and the SIZE of an entry in sectors (could use 64-bit but that would double the size of empty region).
Currently this is done by using 8 bits for size and the rest for offset. This probably isn't optimal but this is how vanilla does it.
Changing that value means that you completely break world save compatibility and all hell breaks loose with existing saves.
So what would be the optimal way? I think it can be calculated. Considering max entry size being 2^SIZE_BITS - 1
, and the amount of them being fixed 32*32
per region for columns and 16*16*16
for cubes (columns are what we are interested in), and the maximum possible size of "holes" between entries (the regions don't support fragmentation) is 2^SIZE_BITS - 2
(if it was one sector bigger, you would have free space to use), and assuming a very reasonable heuristic that you can't really get all of the holes at that size, and roughly half the total hole size will be there, you get total max region size in sectors for columns of 32*32*(2^SIZE_BITS - 1)*1.5 = 1536*(2^SIZE_BITS - 1)
.
We also know that SIZE_BITS = 32 - OFFSET_BITS
. And the max total region size should be equal to 2^OFFSET_BITS
(roughly, it's a good approximation).
So:
1536*(2^SIZE_BITS - 1) = 2^(32 - SIZE_BITS)
for bigger SIZE_BITS we can assume 2^SIZE_BITS - 1 being almost equal to 2^SIZE_BITS
1536*(2^SIZE_BITS) = 2^(32-SIZE_BITS)
2^(32-SIZE_BITS) / 2^SIZE_BITS = 1536
2^(32 - SIZE_BITS*2) = 1536
2^32 = 1536*2^(SIZE_BITS*2)
log2(1536) = 10.58(...) -> 1536 = 2^10.58
2^(32 - 10.58) = 2^(SIZE_BITS*2)
21.41 = SIZE_BITS*2
SIZE_BITS = 10.7
So the optimal amount of bits for size is about 10.7 bits, and considering there aren't going to be THAT many holes in real world, we can round it up to 11. But again that breaks save compatibility.
Instead of changing that you can modify Region
class. What I'm going to describe will be a quick hack and not a properly designed solution.
- add
isExternal
method toRegionEntryLocation
, that returnstrue
whenoffset == -1
- Move
RegionEntryLocation.findSectorFor
toIKeyIdToSectorMap
and make the implementation the default method - Override it in
IntPackedSectorMap
, call super and if size exceeds max supported value, returnRegionEntryLocation
with location-1
and size 0 - in
.IntPacketSectorMap.unpackOffset
, you check for valueOFFSET_MASK
and if it's equal, you return-1
- in
Region
class, this is the "quick and dirty hack" part, inwriteValue
andreadValue
part, you check if the location you gotisExternal
and if so, just read/write to/from separate file inext
subdirectory, named with$REGION_NAME.E$ENTRY_ID.bin
Still happening. Is there any workaround at all (even if it's just expanding the max size)?
[11:49:31] [File IO Thread/ERROR] [cubicchunks]: Unable to write column (766, 7) java.lang.IllegalArgumentException: Supported entry size range is 0 to 255, but got 280 at cubicchunks.regionlib.region.IntPackedSectorMap.packed(IntPackedSectorMap.java:102) ~[IntPackedSectorMap.class:?] at cubicchunks.regionlib.region.IntPackedSectorMap.setOffsetAndSize(IntPackedSectorMap.java:61) ~[IntPackedSectorMap.class:?] at cubicchunks.regionlib.region.RegionSectorTracker.reserveForKey(RegionSectorTracker.java:49) ~[RegionSectorTracker.class:?] at cubicchunks.regionlib.region.Region.writeValue(Region.java:71) ~[Region.class:?] at cubicchunks.regionlib.SaveSection.lambda$save$0(SaveSection.java:63) ~[SaveSection.class:?] at cubicchunks.regionlib.region.provider.CachedRegionProvider.forRegion(CachedRegionProvider.java:148) ~[CachedRegionProvider.class:?] at cubicchunks.regionlib.region.provider.CachedRegionProvider.forRegion(CachedRegionProvider.java:84) ~[CachedRegionProvider.class:?] at cubicchunks.regionlib.SaveSection.save(SaveSection.java:63) ~[SaveSection.class:?] at cubicchunks.regionlib.impl.SaveCubeColumns.save2d(SaveCubeColumns.java:64) ~[SaveCubeColumns.class:?] at cubicchunks.server.chunkio.RegionCubeIO.func_75814_c(RegionCubeIO.java:189) [RegionCubeIO.class:?] at net.minecraft.world.storage.ThreadedFileIOBase.func_75736_b(SourceFile:37) [bgx.class:?] at net.minecraft.world.storage.ThreadedFileIOBase.run(SourceFile:30) [bgx.class:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]