I want to be able to have the mod and also to still be able to join servers. I know its a server side mod, and if you cant make it work on them, can you at least make it so it disables the mod on those servers? I don't want to have to add and remove the mod whenever I want to go to a server...

Which version? The 1.16 version should be able to join vanilla server