The Chalk Brick doesn't have specified tool to break it.
Trying to break it with diamond pickaxe take a while and doesn't drop the block.

Minecraft Version: 1.20.1

Modloader Version: Forge 47.2.0

Mod Version: v0.5.3.1

fixed in 0.5.4 dea180b

seems to extend to slabs and stairs too, will fix for next version!