Everytime the "wings" of the tower are in the fov in-game, my fps's go from ~50 to ~5;
optifine doesnt seem to be the issue as with or without the lag remains the same;
the spikes in optifine's "lagometer" are white;
settings are set relatively low quality (including a render distance of 6);

i have no idea what is happening, but if it is worth anything, the end also has issues with lag when looking at entities (i suppose).

My mod list:
antique atlas;
apple-skin;
arcane archives;
bauble;
betterfps;
chisel;
crimson revelations;
ctm;
embers rekindled;
gravestone;
guide api;
guidebook;
jei;
moreoverlays;
mysticallib;
mysticalworld;
optifine;
patchouli;
thaumcraft;
thaumic augmentation;
thaumic inventory scanning;
thaumin jei;
thaumic periphery;
thaumic wonders;

If you have Optifine, try disabling animated textures. That worked for me (and it's a general FPS improvement in a lot of cases outside of that too)
Big culprit I think is all the glowing crusted stone

Try disabling the shaders (should be under client options) and see if that helps. If not, what CPU and GPU do you have? If you're not sure, the info should be on the right side of the f3 screen.

Edit: to be clear, I mean Thaumic Augmentation's shaders in the mod options