Describe the bug
A spectator player, who was survival before and had a backpack with a magnet in it, can pick up dropped blocks in spectator.
Noticed while inspecting a player on our server in spectator mode, picked up his dropped stuff because I was close by.

To Reproduce
Steps to reproduce the behavior:

1. Be in survival / creative
2. Have a backpack with enabled magnet
3. do /gamemode spectator
4. approach dropped items
5. ???
6. do /gamemode survival
7. check backpack, dropped items in backpack

Expected behavior
No world interaction when in spectator mode.

Versions
Forge: 40.1.76
Backpacks: 3.18.24.704
Core: 0.5.9.115