I don't know how to fix this and why it's happening.
First pic is me not looking at the portal and second pic is me looking at the portal.

Go to the shader's option, turn off motion blur