Whenever simply swords isn't enabled I have perfect FPS but when it is i'm down to 1 This only happens when it is on and i don't use the model fix mod

Model Gap Fix is the only conflict I'm aware of that causes FPS drops.

You will need to perform isolation testing with only Simply Swords and its dependencies to see if you still have FPS drops.
If you don't, you'll need to manually find the conflicting mod in your pack.