the player nametag is visible even when the player is in vanish mode

Are your players in opposing teams?

The default configuration is PVE oriented, friendly towards villagers and other players.
Put players into different teams to allow hurting each-other.
Alternatively you can change this in Spell Engine server config (config/spell-engine/server.json5)

pvp is currently disabled