Forge: 2018
Mod: woot-1.10.2-0.2.3-alpha.jar

for example, T3 looting recipe actually gives you Mass T3.

This is a bit strange, JEI is correct but the actual crafting bench output is wrong. Not quite sure what is going on there, but it will probably be something to do with the enchanted books.

Haven't solved it yet, but I think it is because all the T3 upgrade recipes use the exact same items, except the different enchanted books. If I reorder the books it still gives a valid recipe even though it is wrong.

So I need to subclass IRecipe as I think normal recipe classes don't handle tags in recipe input items.

Sort of got this working ... but there is a catch to my solution.
The recipe will match the NBT tag of the recipe book against the crafting table book.
It has to be EXACT.
I've tested with books from an enchantment table and they seem to be fine. Books from the /give command wont work unless you specify the short correctly.
eg.
Enchantment table book id:21s, lvl:1s
/give @p enchanted_book 1 0 {StoredEnchantments:[{id:21,lvl:1}]}
/give @p enchanted_book 1 0 {StoredEnchantments:[{id:21s,lvl:1s}]}

I will have to look at the enderio code to check but I'm guessing it will be correct.

I think many players use the EnderIO Enchanter to make the books. Would that work as well?

Looks like EnderIO is using ENCHANTED_BOOK.addEnchantment, so it should be fine.

Added custom IRecipe for enchanted book recipes. Tested creating Looting I, II and III upgrades.