primary group changes when switching in between servers. In the database it says I have the group but after I switch to a different server it says I'm back to default.

Check your UUID forwarding settings.
https://www.spigotmc.org/wiki/bungeecord-ip-forwarding/