This is done within optifabric although it looks like they are applied manually by the mod when it loads optifine so I figure it may not be possible.

It is possible but it's a little bit strange. You'll want to do it like

@Pseudo
@Mixin(targets = "net.optifine.OptifineClass", remap = false)
class OptifineClassMixin {

so that it doesn't complain it can't see the type at build time.

OptiFabric does it itself for CustomColours, ReflectorClass and Shaders if you want some examples. You can use normal accessor/invoker mixins to access normal OptiFine classes (to interact with them directly rather than via a mixin) too, not the prettiest but so long as you don't need to touch too much it's a lot easier than trying to compile directly against OptiFine.