I suspect maybe the depth mask material that we add to windows should only render in the deferred pass and not the forward one, and that might solve it. Would we then need two shaders, one for deferred and one not?

Well, doing it only in the deferred pass doesn't work (duh) because it's already written to the depth buffer and the transparent window is at the same depth value.

Probably just need a z-bias like lghassen mentioned