I feel like this would be a pretty common workflow.
If this is something you guys would feel fits into Fabric API, I'll make a PR for it, since I already have a basic Item2ObjectMap-based implementation for it.
(I guess this would fit into renderer-registries?)