Minecraft: 1.12.2
Forge: 14.23.5.2854

Mods: 5
conarm-1.12.2-1.2.5.10
jei_1.12.2-4.15.0.291
Mantle-1.12-1.3.3.55
TConstruct-1.12.2-2.13.0.183
TinkerToolLeveling-1.12.2-1.1.0

Say you set the starting modifiers to 0 and you wanted the user to gain them by other means, such as Tool Leveling. You gain one bonus modifier due to leveling up the Armor & you apply a modifier, say Diamond onto it. If you do this... you are prevented from swapping out materials on your Armor unless you have a material piece that grants an extra modifier or have gained another bonus modifier.

In this image, I have an All Obsidian Chestplate which started out with no modifiers since the config is set at 0. I gained a Modifier by leveling up the armor piece and I applied a Diamond onto it. Now... when I try to swap out any material to something else... such as Cobalt Plates... the armor forge tells me I don't have enough modifiers unless that material piece has a trait that grants extra modifiers.

This doesn't occur with Tools if their starting modifiers were set to 0 & has a modifier applied on it from bonus modifiers.