I defined my friend's house as restricted area,
no one touching it on the blueprints but still got reset from time to time.

And, some buildings/rooms I've been removed from the list also came back too.

I always turned off auto scan option.
Playing on Minecraft 1.18.1
Fabric MCA version 7.2.0 Beta-35
Multiplaying through LAN

I will leave this open for now, but the scanning and restricted area are going to be replaced by a more robust and primitive building system in the near future, solving this issue.

Fixed