Accidentally assigned wrong thing to slot and can't figure out how to fix it...

You can hold Shift to delete a button.
Or you can delete you mine_menu.json in your Minecraft instance to reset it completly.