Describe the bug
buttons like the EMI config button will light up even if the cursor is in your inventory slots.

To Reproduce
Steps to reproduce the behavior:
open a world with EMI open (virtual mouse is on)
check your inventory
move around the inventory with the dpad while watching the EMI buttons
they will light up and the tooltip will show, but hitting A does nothing
Expected behavior
they don't light up at all unless the cursor is actually on them
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Windows 10
• Minecraft 1.19
• Fabric loader 0.14.7 and api 0.55.3
• Mods (nothing except those two interact with the inventory, but i have appleskin 2.4.0, acrhitectury 5.7.28, bedrockify 1.4.2, borderless mining 1.1.4, collective 4.27, ears 1.4.6, emi 0.2.0, guiclock 3.2, inventoryhud plus 3.4.2, lambdabettergrass 1.3.0, lambdynamiclights 2.1.2, midnightcontrols 1.3.0, phosphor 0.8.1, sodium 0.4.2 build 16, xaeros minimap 22.10.1, and xaeros world map 1.24.0)
• Version 1.3.0
• Branch release

Additional context
Add any other context about the problem here.