I tested it with multiple colonies and on different clients with friends. It definitely is MineColonies that is causing this.

Everything worked fine on version 0.2.34.

The frame drop seems to happen every ~5 seconds.

is this client side only?

If you mean that no one else on the server experiences lag if someone walks into a Colonie then yes.

This is annoying! Happens on my singleplayer world as well, update 0.2.35, small village couple of buildings, only 2 guards running around and those micro freezes are just... I'm about to give up on this modpack, not mentioning random crashes while traveling...
Please fix at least those micro stutters.

#edit
So the massive frame drops are gone, just remove the darn Journey map and replace it with Xaeoros, butter smooth fps, the integration with MineColonies is a big issue in Journey map mod.
No idea why nobody in ATM didn't mention about that...

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.