How does Quartz measure Latency?
My Latency based on ElvUI same as the Default Stock UI is always around 20ms. When Casting my Latency shown in Quarts varies alot. I mean I'm talking about 100-500 ms in Quartz while WoW tells me table 21ms.

Whatever the Latency in Quartz is, does it impact Spellqueue?

The latency is calculated by the time between the "spell sent" event and the "spell started casting" event the game provides. Its not entirely clear how it interacts with spell queues and whatnot, but the logic has been the same since its inception.