When a punishing monster deals damage, it treats your armor level as 21-[your actual armor level]
so if you have no armor on, it treats you like you are wearing full diamond+1, but wearing full diamond is essentially being naked.

may need some deliberation on if it can go into the negatives for bonus damage or not, since modded armor often exceeds the armor stat of diamond.