When refreshing chunks on bobby the cached chunks will not load depending on what quadrant you're in.
This is on a vanilla multiplayer server (1.20.6) and only tested in the end.

https://medal.tv/games/minecraft/clips/i2iI9pLd5ZZC81V9U/d1337BLAhPQE?invite=cr-MSw5TlMsMTM2OTY2MDE0LA

Version 1.20.6

After F3+A:

Please try without Nvidium. It's not currently officially supported, and this seems like something it might be at fault for.
If that doesn't fix it, please try without all other mods.