Target dummy is a cool mod for passive leveling of combat but it seems just right clicking it with empty hands racks up the charisma xp :D

Latest ver of pmmo and target dummy on 1.21.1 neoforge

Yeah, the target dummy is an entity. So interacting with it is not different than any other mob. If you have use for charisma, i'd configure the target dummy to give none. if you don't have use for charisma, i'd change the setting in AutoValues to charisma = 0 to turn it off.