Surprised I even figured out what was wrong, for a second I thought it was just going away randomly. If the entity is facing me/looking at me then I can see the plate, otherwise it goes away.

Can you confirm that this is still happening in the newest 1.20 or 1.21 build?

I could not confirm this for 1.20 and higher, so I'm closing this