If you grind, for example, cactus, first batch is 5 green dye. Every additional grind is 1 green dye. If you remove items from output slot, you'll get 5 dyes, and 1 every other again.

Have somewhat the same issue, where the first grind for ore nets me 2 dust, but the next grind only nets me 1.
I can probably make a fix by adding a hopper to the output part of the grinder to fix.