Unsure exactly how possible this would be, but could recipes that require items only checking for specific NBTs? For example:
A player has X custom item, which is an ingredient in Y custom recipe. If the player renames, enchants, or otherwise modifies the X item, it can never be used in Y recipe again, because the NBT data does not match. Would it be possible to have recipes checking for Z NBT data on the recipe item? This might make recipes more compatible, as instead of requiring X specific custom item for Y recipe, it would require vanilla item with Z NBT for the recipe, e.g. The recipe requires "iron_sword" with "custommodeldata: 32" chich would allow for damaged, enchanted, or renamed swords to also work?