Surprised I even figured out what was wrong, for a second I thought it was just going away randomly. If the entity is facing me/looking at me then I can see the plate, otherwise it goes away.

Can you confirm that this is still happening in the newest 1.20 or 1.21 build?