After testing, I found that with both JMX and WaveyCapes on,the cape cannot render properly.
This is a gif showing the problem(link broken)

Hm thats the same visual issue Optifine causes(the reason smooth cape is disabled when optifine is installed). No clue why this happens or how to fix it.

The optifine issue was fixed some time ago, which was a technically illegal usage of the pose stack captures. It should also be fixed here then, reopen the issue if it's still broken.