I tried out this mod to make spyglass a bit more useful, but sadly it doesn't work.
It does lock your slot on spyglass while using spyglass though, so I assume it works at least partially.

Minecraft version: 1.18
Fabric API Version: v0.43.1+1.18
Mod version: v1.0.2 (it's fabric version, I checked)

I have been looking into this a bit. Unfortunately I couldn't replicate this issue. Are you using any other mods?