Barrels are mined very, very slowly regardless of the tool. This is because they aren't tagged with any "mineable" tag.

Playing on Fabric 1.20.1, though this should affect all versions equally.

I can confirm this aswell, however it can be easily fixed by adding the barrels into pickaxe mineable tag through datapack ^^

Same here.