Terrarium

Terrarium

215k Downloads

Remote data updates

Gegy opened this issue ยท 5 comments

commented

All files should use XZ instead of GZIP. Although CPU time is higher, it greatly reduces file size and therefore, IO time.

When the data is being retrieved from the internet, it's greatly beneficial for the file to be smaller, because that will (mostly) be the slowest task. This also applies to hard-drives, where loading from the cache is slow.

The height source should also encode slope into the values with the upper 2 bits, allowing 4 "slope types".

commented

If the height variation in a tile is less than 64 (max height supported with 6 bits), each pixel should be represented by a byte, with a header containing the minimum height. Actual heights can then be calculated by offsetting that value.

commented

The height tile data does not need to be signed, and the tiles can be 1200x1200 instead of 1201x1201. We don't need that extra buffer pixel.

commented

It may be impractical to precalculate slope remotely, as it becomes less accurate when the terrain is scaled up.

commented

I've managed to reduce the size of my local cache from 394MB to 282MB with 401 tiles. This is a 112MB difference! So, this is for sure worth it.

The following optimizations were made:

  • Raster size was reduced from 1201x1201 to 1200x1200, as the last pixel was there for buffer purposes.
  • If total height range can fit within a byte, the raster is now stored with bytes per pixel rather than shorts.
  • XZ compression is now used instead of GZ, which is probably the biggest improvement made here.
commented

All data has been moved over to the new format!