Pretty much what the title says.

Could display it in the menu as having the button for allowing carrying of hostile mobs cycle between [None|Tamed_Only|Any] rather than just [false|true], and for "Tamed Only" use a combination such as B:pickupHostileMobs=false and B:tamedHostileMobExemption=true possibly. (I know that B:tamedHostileMobExemption isn't currently a thing, but it gets the idea across.)

Turns out that some mods (which I will leave unnamed) don't set their added (hostile-but tameable) mobs as non-hostile when tamed, and even if the devs knew how to fix it (apparently they don't?), they would refuse to backport the fix (as they already have with the fix for another more serious bug).

I did try to figure out if similar functionality was possible to do with CarryOn scripting (basically, OwnerUUID is "" for untamed mods and is not "" for tamed ones from what I've read), though that approach seems like it would let people walk off with other people's tamed pets still (assuming that it's even possible to do this with a script in the first place, of course).

Tamed hostile mobs? Which mobs would that be?

Well, in this particular case, the Foxhound mobs added by Quark. You can tame them, they display as tamed in wawla/hwyla, and they behave like tamed mobs. Can even equip chests and wolf armor to the tamed ones with the "Wolf Armor and Storage" mod. But they still show up as hostile on minimaps and can't be picked up with CarryOn (because "hostile").

Ok, I see now. Yeah, I think I can probably add that.