I have tested if it was something else in the modpack by using a fresh install with just the Simply Swords mod installed.

Using a runic tablet in the smithing table with a runic weapon to reroll the ability as it says it should do (even without shift clicking as the info page says to do) does not reroll the ability.

• Modloader: Neoforge
• Modloader Version: 21.1.169
• Minecraft Version: 1.21.1
• Mod Version: 1.60.11
• Environment: Both Singleplayer and Multiplayer (plus seperately with only the simply swords mod installed)

Fix is now live ^

Fixed in #255. A fix to this will come in the next update.