I base in caves a lot, so the walls are made of Stone. But so is the ground underneath the floor and above the ceiling. When I chisel the walls, it also chisels the entire surrounding caves which is super annoying and takes more durability.

This is intended behavior of that mode, but I have considered adding one that only changes surface-level blocks, or altering the behavior.