When you use multiple Fortune glyphs on a damaging glyph; such as Harm, Cut, etc; it only acts as though you put a single fortune glyph.
Notably, Fortune works fine on Break. Appears to work on Harvest.
When tested with Fell, it didn't seem to work, might be the same bug, might just be the limits of my quick and dirty testing.
Mobs were tested extensively, including using commands to make a looting 8 sword. Results with any non-zero number of Fortune glyphs matched a fortune 1 sword.
Drygmy fortune boost seems to work as intended.

A quick test with just Ars Nouveau and its dependencies showed no change in behavior.

What mobs were you testing on? I am having trouble recreating this.
I have created a harm + fortune 8 spell, and cows are definitely dropping past their cap consistently.

Fortune works on a curve system, and every mob has its own unique looting effect. Many minecraft mobs do not drop additional items from looting, and instead just gain an increased chance to drop per level.

The issues isn't that fortune doesn't work at all, but that spells don't benefit from more than one stack of the fortune glyph.
I did some more rigorous testing with many different mobs and a decent sample size.
Each spell was Touch->Harm->Fortune, with the only difference being the number of fortune glyphs.
Tested with chickens:
With 128 chickens spawned:
1 fortune glyph: 256 raw chicken, 249 feathers
8 fortune glyphs: 256 raw chicken, 256 feathers
With 128 creepers:
1 fortune glyph: 259 gunpowder
8 fortune glyphs: 258 gunpowder
With 128 zombies:
1 fortune glyph: 254 rotten flesh, 1 carrot, 2 potatoes, 5 iron, 1 item drop (iron shovel)
8 fortune glyphs: 240 rotten flesh, 0 carrot, 2 potatoes, 2 iron, no item drops
With 128 Wither Skeletons:
1 fortune glyph: 261 bones, 8 wither skulls, 89 coal, 14 swords
8 fortune glyphs: 276 bones, 8 wither skulls, 95 coal, 8 swords

Mods: Ars Nouveau 2.3.3, forge 40.0.34, Curios 1.18.2-5.0.7.0, Patchouli 1.18.2-67, and GeckoLib 3.0.15
This testing was completed on 04/18/2022.

Hope this helps!