I'm using 1.14.4 Fabric build 12, Loader 0.4.8 build 159 with the following mods:

Fabric API 0.3.1+build.208
Immersive Portals 0.12

As I move around, the point from which the portal sees from moves as well, causing it to clip through blocks and see things it shouldn't from it's intended perspective.

I fixed culling with shaders but accidentally broke the culling functionality without shaders. It's fixed. It will be in 0.13 version.