If you make a world called "new world" and then delete it, then make a different world called "new world" it will use the leftover preview from the original "new world".

You are right that previews are saved with the world name. Reason's so that users could easily find the image file to copy it if they want to (located under the screenshots directory)
I'll fix this in the next update by removing the image save when its associated world is deleted. This'll only apply to singleplayer worlds, where deleting a world usually means you won't be able to come back to it. Thanks for the report!