I honestly don't exactly know when it happens or how to replicate it, i think when minecraft is not focused

Try the GLFW replacement test for Windows (feedback is welcome). It fixed similar issue for me.