A symptom of another issue I was experiencing (which is hopefully now fixed) somehow managed to create over 1 million region files in the world save:

Upon attempting to delete this, the entire game hangs, because obviously, this is a massive count of files. I know this is an extreme example but I imagine it still has to wait a bit for larger/pregenerated worlds.

I recommend doing the operation on a separate thread such that it doesn't hang the entire game, and providing a progress bar so it's possible to actually see what's going on and what file is currently being deleted.

I imagine you could overlay the progress bar and current file onto the world's entry in the saves list (and immediately grey out the world entry once the operation is started so you can't interact with it anymore).