There should be a flag to disable/enable destroying and build flag only controls building not destroying.

I have public mines but i don't want players to establish their home in the mine (Or place chests and stuff)