VanillaFix has a dynamic model loading system, which reducec memory usage to 2-3 GB. I tested it in a singleplayer world but it crashes after a minute.

Could it be beneficial to explore possible compatibility with VanillaFix?

It's too unstable I'm afraid :/