I noticed the book's statistics weren't updating when I was punching wood until I took my shield out of my offhand. A minor complaint but nonetheless irritating if you don't notice it early on and are trying to get the count up for progression related reasons.

Minecraft 1.19.2
Forge 43.1.52
Primal Magick 3.2.1

This has been changed in the upcoming 1.20.1 release. You should be able to gain progress towards breaking blocks barehanded even if you have something in your offhand, such as a shield.