I added ActuallyAdditions 1.8.9-r26 to FTB Unstable 3.0.42 (for MC 1.8.9) and most of Thaumcraft 5's item graphics became squares of randomly colored static, as if the system had used the wrong bit of data for the graphics.

Please describe it more specifically.
Did you use JEI or NEI? What things were mangled when looking at what? Did you have anything else around from any mod that rendered in any specific way?

I'm using the JEI that comes in the pack (2.28.7.174). I'm putting in a shot of JEI and of the view of the Thaumcraft research table. Everything looked normal prior to adding AA, and has remained like this after multiple restarts (SSP world, no separate server).
The only graphical problems I found were in Thaumcraft, except for one of the two Thaumic Infusion items.
Oh, it should be noted that I picked this page in JEI because the essence phials had their essence animations rendered over the mess, although you can barely make it out here.

I've discovered that Quantum Flux has the same effect. I've notified Azanor and joekaiser in the hopes this can be more easily resolved.

Did you get any new information on this?

Azanor said it was a mod pack issue and closed it. I tried a custom pack with just TC5 and a couple of others and didn't have the same problem. joekaiser gave me some good information though. F3+s reloads the texture pack, which fixes the graphic until the game is reloaded.