Would there be any way to add a configurable option to add/change the Affinity or Aversions, for gems like lapis lazuli who have none of both, or for modded gems like blue topaz from ars magica or amethyst from biomes o plenty etc.

The latest commit added Minetweaker support for gems, that feature will be in the next release