Reproduced on 5166163.

I ran two tests with a standalone instance, with a render distance of 2, followed by a render distance of 16. Right clicking on the Observatory and while mounted, appears to drop the FPS significantly.

video log

With the render distance elevated, the FPS drop is much more prevalent. Running 80-100 FPS usually in the Overworld - Sitting on the observatory drops down to 30-40. Will report back if there is any additional information with this interesting event.