like either make baubles hotkey close guis like E does or make baubles inventory render on top of minecraft inventory or to the side of default inventory

It is possible, but it is almost guaranteed to break other mods that also alter the GUI which is why it works like it does.