i got the https://www.wowhead.com/item=229953/salvaged-goblin-gazillionaires-flying-machine like a hour ago, and the achievement still on the screen, clicks dont remove, the only thing it does is go up or down if I Looted new things.

addon version: r 745

This might be a problem with Blizzard's achievement UI, which Rarity uses. If you mouse-over the widget it should fade out again.