![Terrarium](https://media.forgecdn.net/avatars/thumbnails/241/29/256/256/637125232933085085.png)
Remote data updates
Gegy opened this issue ยท 5 comments
All files should use XZ instead of GZIP. Although CPU time is higher, it greatly reduces file size and therefore, IO time.
When the data is being retrieved from the internet, it's greatly beneficial for the file to be smaller, because that will (mostly) be the slowest task. This also applies to hard-drives, where loading from the cache is slow.
The height source should also encode slope
into the values with the upper 2 bits, allowing 4 "slope types".
If the height variation in a tile is less than 64 (max height supported with 6 bits), each pixel should be represented by a byte, with a header containing the minimum height. Actual heights can then be calculated by offsetting that value.
The height tile data does not need to be signed, and the tiles can be 1200x1200 instead of 1201x1201. We don't need that extra buffer pixel.
It may be impractical to precalculate slope remotely, as it becomes less accurate when the terrain is scaled up.
I've managed to reduce the size of my local cache from 394MB to 282MB with 401 tiles. This is a 112MB difference! So, this is for sure worth it.
The following optimizations were made:
- Raster size was reduced from 1201x1201 to 1200x1200, as the last pixel was there for buffer purposes.
- If total height range can fit within a byte, the raster is now stored with bytes per pixel rather than shorts.
- XZ compression is now used instead of GZ, which is probably the biggest improvement made here.