The title explains the issue.I think this would be really helpful as you can remove the diamonds from players inventory when you catch them xraying.

I'm pretty sure that's a minecraft issue. This is not possible to fix.

When in spectator mode, you cannot edit any inventories (such as chests and player inventories) without the use of commands like /clear and /give.