The map is empty, and log spams message "TileGroup tried to set tile out of bounds".

This can be solved by making tile coordinates int instead of short. However, network packets currently write short. In case of a version mismatch between client and server, the client will read garbage. Therefore for longer coordinates a new packet is needed.

Premature optimization is the root of all evil.

This is still a thing in the Fabric version. I need to cherry-pick that commit and massage it until it works there also.