The dirt slime only takes a half heart of damage which is the same as punching it.

Yes, it has damage reduction because being made from dirt gives it good armor. Try a better weapon; if I remember correctly a diamond sword should make 3 points (1.5 hearts) of damage.