When I understand it correctly, I can disable, that torches will be placed when my hand is empty but I have a torch in my offhand. Where can I toggle this? (using forge on 1.16.4)

Tweaks can be enabled in the config/clienttweaks-client.toml on recent versions. Forge has not yet re-added the mod config screen.