I tried to implement a set with a potion effect that increases in potency each rank. However in doing so, once the set is equipped, the lowest set bonus is replaced with the strongest version of that set until the world is reloaded and the set piece is re-equipped. Of course, if you re-equip more pieces, the lowest set bonus changes again.

Example:
2/2 Slowness I
3/3 Slowness II
4/4 Slowness III

Works correctly incrementally. Once I equip 3/3, 2/2 permanently transforms into Slowness II. Once I equip all 4, 2/2 becomes Slowness III. The 3/3 bonus does not change.

I experimented further by adding a 1/1 set which was just Slowness 0 (which defaulted to 1 anyway). The same effect was seen although the 2/2 Slowness I remained unchanged. Of course, this still meant that Slowness III took precedence anyway.

This effect is not visual - it does alter the potion effect applied.
It does not seem to affect attribute stacking, but I didn't test that very thoroughly.

Thanks for the report! Someone mentioned this on the curseforge page previously, but I've been putting it off due to a half-finished feature gone wrong.

Hopefully I'll get this fixed no later than this weekend. The feature I mentioned can be cut off for now instead of eating up my time and motivation in regards to this project :P

This should now be fixed in the latest release (1.12.2.009)