Off-by-one in denormalizeShort
falkreon opened this issue ยท 4 comments
In https://github.com/jellysquid3/sodium-fabric/blob/1.16.x/dev/src/main/java/me/jellysquid/mods/sodium/client/gl/SodiumVertexFormats.java#L97 Sodium converts java floats to unsigned short representation, using (short) (value * 65536.0f);
This is incorrect. Per Khronos docs ( https://www.khronos.org/opengl/wiki/Normalized_Integer#Unsigned ),
For a given integer of bitdepth B, the maximum representable unsigned integer is MAX=2^B-1.
Which in this case is 65535, or 0xFFFF. If we leave the code as-is, 1.0f will encode to 0x10000 instead of 0xFFFF and cause some rare rendering bugs and extremely minor misalignments.
Good catch. This fixes a number of other z-fighting issues we're suffering from as well.
Fixed with jellysquid3@a7f058b.