Minecraft Version: 1.19.4
Fabric Version: 0.14.19
Fabric API Version: 0.79.0
Framework Version: 0.6.4
Backpacked Version: 2.2.0

The server doesn't crash, just the client. I do have a backpack on and that's when the crashes started happening, I'm not sure if its an incompatibility with Immersive Portals or not but here's the crash log. It also seems to bring up the Framework API , so it could be something there as well.

It was caused by Framework. Should be fixed if you install a newer version