Optifabric conflicts with lambdacontrols, a mod that adds controller support. The game crashes on startup. I would love it if this got fixed!

LambdAurora says she won't fix this btw

Just as a note I think it was the overlay which didn't work with OptiFine, it's LambdaControls which is reporting itself as being incompatible though so this will need a change in that once fixed anyway.