On one hand, a 40 MPH crash into a brick wall probably won't blow it up, and if the car is built well enough you might even be able to drive away from that.
On the other hand, you'd probably walk/drive away with whiplash, or something else from the sudden stop.

While I can see this being a plus from a broad overview, doing something like this will get complex really fast. For example: how many brick blocks must one hit before the car crashes? Does the player need to be close to the crash area? What about wood? What about clay? How will the player respond if they're in the back of a bus that hits a brick wall? Will they even notice, or will they think they got hurt randomly? All these questions need to be addressed and answered, and given the small playerbase on survival mode, I just don't see me putting in all the effort to do so.