I was messing around with signs and ended up right clicking them with a wrench. When I did this, the hitbox shrunk down to just the pole, and when you destroy the pole the sign remains. I don't know if this is a intentional feature that's bugged or just a bug itself, but I can't find a way to destroy the signs. Also keep in mind this was discovered on a server, so it might be different in single player.

Here's an example image of the bug:

Fixed in 20.3.0.