Seriously, the button below your inventory that allows profiles is waaaaay too big and it would be nice to have a way to turn that off.

I'm assuming you are talking about the button that switch to CosmeticArmor Inventory GUI.
It's size scales with GUI, it shouldn't be too big unless your modpack have too many buttons added to player inventory.
You can customize it's position or hide them in cosmeticarmorreworked-client.toml.