Now, I am not sure if this is a good idea seeing as these huts are in basically everywhere, but it would be cool none the less.

BP 0.3 will probably significantly lower the overhead of loading (not necessarily of rendering) multiple portals leading to vastly different place (as long as they're all in the same dimension) and maybe even allow for variable loading-distance in portals (i.e. the closer you get to the hut, the more on the other side of the portals will load).
Once both is supported, I'd consider TravelHuts integration viable.