Edit: I've both updated my graphics drivers and tried to play around with the config
Not only does it not fix the issue, I also can no longer replicate the result of the bug fixing itself after a few minutes

Version: 1.18
Most of the time the world only loads inside the simulation distance
A lot of the times, after a random interval (usually more than 5 minutes from loading the world) it will fix itself and start loading outside of it
I've removed all other mods and tried 4 different game versions and 2 different api versions but the issue persists
All my settings/config are default

An unnecessarily long video of me showing off the glitch:

I also noticed such a bug
on version 1.18 with c2me mod

Fixed in 01c5bc1

Try again with latest dev build and see if it is fixed.