Why do you have those fancy buttons on your mouse, if you can't even close your inventory with them?
This is a fix for that.
Also the issue link to Mojang's bug tracker: MC-577
client-side only,
Fabric release should be compatible with any Minecraft version.
For other fixes, see Blanket
NOTE: Blanket include this fix.