Im wondering why the Shadowmetal blade, something that can be made much earlier than the Rainbow Sword does 20.75dmg compared to the 18.5 of the primordial blade. Wouldnt it make sense for them to be swapped or atleast buffed?

You have to keep in mind that the fortress blades CAN NOT be enchanted via any means. You can force enchants on the fortress blades with creative or NEI but the enchants have no effect at all on the weapon.
rainbow sword can be enchanted quite heavily.

SFB no enchant: 20.75 damage
SFB with sharp & wrath V: 20.75 damage
rainbow sword with sharp & wrath V: 31 damage

Fortress blades are supposed to have more raw power while the primal sword has the draw in effect. Moreover, the fortress blades cannot be enchanted while the primal sword can.