I noticed that when I hover over the gold count in my inventory, it shows a number of characters that have since been deleted. After some testing I found that it was the BagBrother add-on. I found the .lua that seems to hold the data, so I assume I could just manually delete those characters, but there doesn't seem to be a way to handle removing or inspecting character data through the options in-game.

You can remove a character using the "character" button in the top left corner of the bag frame, that you normally use to look at other characters' bag contents. When you hover over a character name, instead of clicking it, a sub-menu will appear with a "remove" option.