So apparently weird lag spikes happen when updating dynamic light that is caused by a player holding any item (player moving). That happens only on surface (possibly when interacting with non-full block?). That doesn't happen in the cave where there is only stone, and also high in the air where there are no blocks at all.
I'm playing Minecraft 1.20.1, Forge ver 47.4.1.

What happens on the surface:

And what happens in the sky or a cave:

Maybe it's some weird mod incompatibility?