There is a bug with view bobbing that only happens when you look at an immersive portal.

When you look at an immersive portal the view bobbing will completely stop
When you are looking at a portal thru a wall the view bobbing will go crazy like if there is an earth quake.

Here is a video that shows the bug https://youtu.be/Y862-LwFcng

It happens with and without optifine but with opitfine you notice it more because optifine disables arm sway.

Probably fixed.

This is not a bug, this is on purpose. Direct quote from qouteall:

Because of the camera transformation. View bobbing applies on camera transformation, it contains rotation and translation. When rendering a scale portal I does not apply the scale transformation into the camera because it will make the fog abruptly change when crossing the portal. Then the view bobbing's translation applied onto camera transformation is inconsistent with the outer view bobbing

I am going to make that disabling view bobbing does not affect hand movement