I've noticed a really jarring issue with this mod when you enable dimension stacking. From what I can tell it's definitely only Immersive Portals and it's on the fabric version. I'm not sure if this matters but I am also on a server too. When you enable any sort of dimension stacking at all, there's little spots (seemingly random) in the world where your camera in both third and first person will start shaking violently when moving. After numerous testing, I've seemed to narrow it down to if you have view bobbing on or not. Once I turned that off, the shaking stopped entirely. Back on, it starts again.

Besides turning off view bobbing/disabling dimension stacking, is there any way to fix this with the mod currently or will it have to be some sort of bug fix?

Are you using optifine? The view bobbing suddenly stops when viewing portal when optifine is present

I am using optifine. To be honest I didn't realize that would affect it so I didn't try uninstalling it. Kind of dumb I didn't since there's a message in chat mentioning Optifine and Immersive Portals. That's my bad.