I found a bug! The map HUD causes Lag spikes (not FPS, network latency). According the the friendly neighborhood task manager, my network usage goes from about 70Kbps to 5000Kbps when I move the map from my hand, to the little inventory box.

I'm playing on my own server with TheCampingMod_2.1g.jar. I don't think this is a bad interaction with another mod, but for what it's worth, the modpack is here: http://technicpack.net/modpack/nomads.657571/permissions.

I'm sorry, but I can't seem to replicate this. Can you try and see if it still occurs without the other mods?