Problem

We are starting progression on a new boss, everything is explained, and the raidleader links a WeakAuras-display in chat. "Get this, it contains everything you need". Everyone clicks on it, and we have to wait five minutes to actually pull the boss, because everyone is sent their own personalized message.

An example scenario and what would change

1. Anna links her display in chat.
2. Bert clicks on this link, requests display.
3. On Anna's side, the first "text" is sent the same way as before, but we now also add it to the "send-cache". See sample code below for what exactly "text" is.
4. Bert receives the display in the same way as before.
5. Charlie now also clicks on the link, after Bert, but before the transmission to Bert finishes.
6. On Anna's side, the same display should generate the same text, so it's found in the "send-cache". Anna now queues sending "text" again, but this time addressed to everyone in the raid (because Charlie is part of Anna's raid group).
7. The first transmission (to Bert) finishes.
   1. Bert fully received the display. The popup shows to import the display (same as before). He also puts "text" into his "received-cache", along with a note that its popup has been shown.
   2. Charlie may also receive it, but discards it as it's not addressed directly to him (same as before).
8. The second transmission (to ALL) finishes. Everyone (Bert, Charlie, and possibly Anna?) unpacks "text" and adds it to their "received-cache".
   1. Bert sees that "text" is already in his "received-cache", and the popup was already shown, so nothing else happens here.
   2. Charlie has not seen the popup for "text", and he requested something from Anna earlier => show popup, and set status of "text" to "shown".
   3. (only if Anna also receives the 2nd transmission) Anna sees that "text" is part of her "send-cache", no popup shows up, and status of "text" in "received-cache" is set to "shown".
9. Two minutes later, Daniel comes back from his break, sees Anna's link in chat and clicks on it. He already received both transmissions - the first one was discarded, and the second one put into his "received-cache" with the note "not yet shown". The latter matches both the sender-name and the display-name he requested with his click, so the note is changed to "shown", and the popup is displayed without any delay.

Both of these caches are cleaned up every few minutes (I suggest 5 minutes like the compress-cache). If these caches are empty, the behaviour is the same as before.

"send-cache"

The "send-cache" would only decide if we send something to the whole raid, or just a single player.

Contents of the "received-cache":

• key: "text" (as is transmitted)
• value: {
  arrivedTimestamp, -- 1234567890
  sender, -- "Anna-Samplerealm"
  hasBeenProcessed -- "no" / "yes" (maybe more, maybe call it "status" instead)
  }

The "received-cache" would be checked every time we click on a link. Whenever player A requests a display from player B, the "received-cache" would be searched for any messages of player B. If there are any, they are processed as if they arrived just now. When one of the messages contains a display, we should only show the popup if the display's name matches the one we clicked on, otherwise we discard that message (set status to "processed"). We need this because some people could have requested multiple displays of player B, and we could have them all cached on our side. Note: I'm not sure but I think this can occur even now, we click on a link of player B and they can send us anything, because we don't check for display names after that.

TODOs and possible problems / edge cases

1. We would probably need a table with ["person we requested display(s) from"] => {"name of display 1", "name of display 2", ...}. Fill on chat clicks, empty when one of them is shown as an import-popup / non-exist-response.
2. We put some display from Player B into our "received-cache", they change something or fix a problem with it and link it again in chat a second time. This time we click on it, but the old version is still in our cache. This would all have to happen in the timespan an entry in the cache is considered "fresh". Is that something that can be ignored? We mark that entry in our cache with "popup shown" anyway, so the player would have to close the popup and click on the link again to get the new version. Possible solution: Send a quick "remove everything from received-cache that's from me" message when linking in chat. Or the other way around: Player B links a display in chat => remove all cached messages from that player. That would at least be no worse than before.
3. One complication I'm seeing now is one that's not yet solved at all in WoW's SendAddonMessage, and won't be until there's some sort of transmission control happening (like TCP). When party A starts sending multiple messages to B, and B gets a loading screen (e.g. clicks on a portal), then B will only receive the first and the last messages, but everything in between will be lost. I have not spent time on this issue in a few years, maybe it has been solved already. If it has not, then we would need to safeguard both inflation and de-serializing of incoming messages. Think of Daniel zoning into the raid instance sometime during transmission 2 in the above example. As a side note: This problem also occurs right now, when I click on a link in chat, the transmission begins, i zone into the raid, and the transmission ends.

see: https://github.com/WeakAuras/WeakAuras2/blob/d3ab05b17bcbac6cb732f5f2f635be1f5c9e8216/WeakAuras/Transmission.lua#L1652
(added code as comments)

local function crossRealmSendCommMessage(prefix, text, target, queueName, callbackFn, callbackArg)
  local chattype = "WHISPER"
  --local pathChosen = false
  --if isInSendCache(text) then
    --if target in same raid [or party] as self then
      --addToSendCache(text, time()) -- refresh cache
      --chattype = "RAID" [or "PARTY"]
      --text = ("§$ALL:%s"):format(text) -- send with everyone as target
      --pathChosen = true
    --end
  --end
  --if not pathChosen then
    --addToSendCache(text, time()) -- send to cache
    if target and not UnitIsSameServer(target) then
      if UnitInRaid(target) then
        chattype = "RAID"
        text = ("§§%s:%s"):format(target, text)
      elseif UnitInParty(target) then
        chattype = "PARTY"
        text = ("§§%s:%s"):format(target, text)
      end
    end
  --end
  --cleanupSendCache() -- remove stale content from cache
  Comm:SendCommMessage(prefix, text, chattype, target, queueName, callbackFn, callbackArg)
end

Variable names obviously placeholders just like the pseudo-code itself.

What do you think about this?

And to expand a bit:

This is a niche case, that only affects a very small niche use case, which is also easily covered by other sharing mechanisms. If you send a PR, I'd obviously consider it and if suitable merge it. And you can also ask questions on how it should ideally work.

BUT!

If wago lib is ready, (which we don't know if/when that'll happen) I think we would switch WA to that and remove the old transmission sending code asap.

Wagolib uses a smart sharing system which should cover this use case.

So, I'm closing this, since as far as I can see no one on the team wants to work on this. Although if you have questions on how a better sharing mechanism could work, feel free to ask. Or even better write a PR so that we can discuss it. Or wait for wagolib to be used.

My biggest concern was time spent sending something over the addon channel, which cannot be circumvented with sequential transmission and a single sender. Compression times could in theory always be solved with a better PC ;). This WagoLib looks promising, and seems to aim at mimicking torrent behavior. Although I'm already thinking of ways to interfere with and break transmission, but that's not part of this issue right here.

So thank you for the links, I will definitely monitor them, and maybe contribute if I find the time.

Leaving with another thought:

Have you guys thought about caching the serialized and compressed version of any display right after anything is changed, or maybe right after a chat link is created? Player A edits their display, creates a chat link and presses send. Then WeakAuras creates the compressed display-string. Player B and C both request the display, and player A already has everything they need to create the outgoing message(s).

As to the last comment, creating the compressed display on sending a link is something that could be done. It would require a PR from someone that cares enough about it, as it is a rather small improvement that likely doesn't really matter.

Reworking the general mechanism how auras are shared is a much bigger undertaking. I'm certainly not against sending auras via the raid channel.

I certainly have no interest on working on that though, since there's nothing that can fundamentally fix how slow the addon channel is and that sharing via a outside mechanism is far superior.

More efficient transmission (read: doesn't brick your computer when 19 ppl click on a link simultaneously) would definitely be valuable. I believe there is some effort going on here that you might be interested in looking at, though i don't know of any ETA.

Also you may want to track rossnichols/LibSerialize#4 & SafeteeWoW/LibDeflate#6.