When I'm using the mod on my Server, moving the Sliders is really glitchy. Say i want to drag it all the way to the right but don't have enough XP it resets back to the beginning.

Thats the only way too to get there. I can't move it back there manually.

Using it locally seems to be fine. Especially in creative mode.

Using newest version on Forge 2099 and 1.10.2

Yeah I notice this quite a lot, it breaks the tactile-ness of the mod and makes the gui feel kinda mushy, would be better I think to allow scrolling, but say you don't have X to enchant.

This issue has been fixed in 1.12.2. The sliders currently feel a bit stiff, but this will probably be fixed before a public release is made.