I'm not sure how i'd go about this but it would greatly improve rendering and performance

This would make it so we'd have to do more checking than the current wasteful system. I'm going to close this for now because I think we'd not actually see any improvements with this sort of change being made.