While messing with the new action bar setting that were released with the update, I noticed Action Bar 4 and 5 were flipped on the customization tab. So the setting changes on the addon for Action Bar 5 would change the in-game Action Bar 4, and vise versa.

Settings should not be flipped, but order changed on the big settings update to match retail wow button order.
Will close, but feel free to open a new issue if you encounter anything unusual.