Below is a video that shows the following:

I add 2 talent points to my "Raid..." build and apply it, then update the tree (the button next to apply or the one I pressed doesn't matter).
I click on the "Raid..." build to show it's active and currently applied.
I click on the "Questing" build which loads fine.
I then click back to the "Raid..." build and while it does change the choice node in the class tree and switches the spec tree properly, it fails to reallocate the class tree points I just updated.

https://drive.google.com/file/d/1mutSjceI1BXjDuhmYpwpPxLLbc7YjxtZ/view

I thought I fixed it by replacing the old loadouts by new ones, but that only fixes it for 1 apply, then it breaks again. If you need more information, let me know! I deleted all my loadouts but those two at this point and it's still happening.

You are saving the tree to a spec loadout. Spec loadout only change the spec tree even though they also have the info of the class tree.