It's not a hard incompatibility but the game and world load is much slower with both installed.

Not sure if that's just something that is guaranteed to happen?

Pretty sure this is because Smooth Boot forcefully caps the number of worker threads that the game can use during launch. This makes sense in vanilla (where very few threads are needed due to not many resources being loaded), but in large modpacks it is beneficial to be able to use all the threads.

This is normal, also happens in vanilla not with ModernFix only, as Smooth Boot (Reloaded) tweaks the number of the worker threads the game use.