I've found that the piglin charm simply doesn't work, be it in my inventory or on a charm belt, killing them damages it but they are still otherwise hostile.