vanished players are counted in server number
when a player does a ping of the server (from the serverlist) and is logged in at the same time can use current player list - count to get the list of vanished players.
A mitigation tactic: only show nearby players in the playerlist (minus vanished players).

I have the same issue. Players join the server from the Minecraft browser knowing how many players are in the server, but then are confused when no players are present and realize there is a vanished admin.

Put ProtocolLib on and it goes to the number minus vanished.