This is due to my bad chunk rebuild code that tries to do everything at once, rather than spreading the rebuilds across multiple frames. I'll be fixing this soon!

Fixed by rewrite!