There is a mod, Magic Yarn, that does this well.

Single player: uses the item
Server and Client: uses the item
Client but not server: uses a hotkey
Server only: not required on client