If an item is picked up, you could replace it with another item in inventory and the star icon will not follow the new item, instead they will stay on the original slot.

Using: 1.21 Fabric with latest Item Highlighter and Fabric / Fabric API.