Spigot 1.9.

When you remove citizen they disappear but can randomly appear by themselves around the map (on server restarts possibly?) it seems skRayFall doesn't remove them from Citizens config --> Citizens recreates the supposedly removed citizen on restart.

If you delete a citizen it should be removed from the "saves.yml" file. Try using:

(remove|destroy) citizen %number%

If this doesn't work please reply with your servers full startup log so I can see what is going on.