Version for 1.20.4 is now released on Modrinth.

Pufferfish's Skills 0.11.3 is marked as 1.20.4. But while it doesn't crash in main menu, it gets stuck at world load.

That's because I have incorrectly set game version for that mod file while uploading. It is fixed now.

I also have the exact same Problem, as soon as i switched my Version to 1.20.1 (I had two other Mods i wanted to combine here and had the exact same Problem) the tend to fade away. With this version i no longer run into crashes while loading a world. I know it's not a fix for the 20.4 version but for me it helps a bit, i did an Backup of my World in 20.4 and i'm trying these mods on 20.1 for now on.
Maybe it helps you! Stay healthy and good luck.

Pufferfish's Skills 0.11.3 is marked as 1.20.4.
But while it doesn't crash in main menu, it gets stuck at world load.