When a non-household sim is taken into CAS and the game then returns from CAS afterwards, the household of the selected sim will be loaded instead of the originally active one.
This is caused by the game always loading the household of the sim which is currently edited in CAS.

Since CAS seems to be pretty much disconnected from the script server, I'm not sure if this is something that can be fixed.

This is a bug in the Sims4 client code by Maxis and can not be fixed by anyone other than them.