Hi! We'd talked briefly about this over in your reddit thread. The idea stuck with me so I kept poking at it. I started writing a
scraper for dataforazeroth to get you the source data and then stumbled across this:
https://github.com/justinsmid/data-for-azeroth-scraper
It dumps the mount name and % of users in a clean text file. Worked flawless and was honestly much better than mine. :)
data.json.txt

Yeah the author of that did it as part of the same reddit thread:

https://www.reddit.com/r/wow/comments/odbjjk/happy_10th_birthday_litemount_ten_years_ago_today/h4193e1

To which I responded:

This is very cool, but the requirement for a full chrome is a bit heavy weight. What I'm looking for is something that runs in a github action, since I don't have any hosting nor do I want to do it by hand.

Actually what I really want is someone else to make a site that produces this table (though actually I need the mount ID not the name to work in other locales) every hour or every day. Then I can put something in LiteMount where you can import it, and people can cut-and-paste it in.

Shooogen also made this: https://www.dataforazeroth.com/collections/mounts.json

which removes the need for the scraper, but still requires a full browser to interpret the javascript, so isn't automatable (by me anyway).

I haven't forgotten about this, I did start work on a priorities import. I don't have a lot of time right now and also there are a couple of serious problems behind the scenes with the rules I need to iron out.

The basic import was turning the percentages into the 1/2/3 priorities, which although not much granularity is still not bad. Hard to know what to make the cutoffs though.

Right on. I might be able to help with the fetching/hosting part. Or maybe better I could just do a scheduled git push for it. Do you have a format in mind that would make it easier? Maybe something like a csv: "Mount ID",percentage

No worries about the time, it's all for fun right. :) Appreciate all you do.

I recently discussed this with a few friends and ended up implementing it in a very basic manner (don't judge, my first add-on): https://github.com/sgade/RandomRareMount. It uses hard-coded data from the site you linked above and can work with their given percentages exactly.

I found that I would like to keep using LM, though, for features like its smart use of shapeshifting forms.
So before I create a PR, I thought I'd chime in to the discussion.

A few things came to my mind when reading this issue:

1. Integration into the existing 1/2/3 priorities can be done but looses a lot of the granularity of mount distributions. The mapping should probably be done in discrete ranges (e.g. 0–5% -> 1, 5–20% -> 2, 20–100% -> 3).
2. I wanted to automate refreshing the mount rarity data anyways using GitHub actions. It's a bit hard because they seem to be static files with a hash attached, so the filename obviously change when the data changes, but there probably is a way to get the filenames from requests alone without executing a full-blown V8 in the pipeline.
3. The pipeline could either commit the new data only, or additionally also release a new version.
   Releasing only for new data could become a bit spammy, though (cf. Raider.IO's add-on). At the same time, with mounts like the crawler from this month's trading post the data is changing quite a bit in a short time because many players will be able to get it.

I could try to make it work for my add-on as some sort of proof of concept and we could discuss how you would like this to be integrated into LM. What do you think?

Hi, how exciting.

First an apology. I probably won't have too much time to look at this for a while. Real life is kicking my ass so everything else is a bit background rightnow.

With regards to your points here's my initial thoughts:

1. Don't want to integrate with the priorities, I don't think they interact in any useful way. I think I would rather have this be a separate action RarityMount (or something) that ignores the priorities completely and does the right thing.
2. I'm not keen on spamming out LiteMount releases for priorities. But I think this can be done sensibly with a separate addon (e.g., LiteMount_RarityData). LiteMount can have a baked in set (or maybe none, not sure) of rarity weights, an OptionalDependencies on the rarity data addon, and use the RarityData from the separate addon if it's available.
3. That way the LiteMount_RarityData can be updated very often (even daily). It is only installed by people who actually care about updated rarity data. And, even better, you can maintain it instead of me. :)

Edit: perhaps it should still check for priority 0.

Looking at this a bit further, SmartMount probably seems ok and pointless duplicating. Perhaps it would be enough to have a function to replace PriorityRandom with your weighting random function.

I've committed an empty RarityRandom into the code and a half-hearted hook for it by looking for a run-time setting. I'm not sure if it is better as a rule that switches the weighting type or a global option. You can just comment it out for your own testing and I'll worry about how it would be activated later.

So what I would accept for a PR I think is (a) a filled in RarityRandom function in MountList plus (b) an OptionalDependencies in the ToC to whatever your rarity weights data addon is. No point trying to embed the weights data into LiteMount itself at this point (if ever).

Another thing you will need to consider is how to handle non-journal mounts (forms, items, etc) that will not come out of your weighting data unless you do something to fake them up.

I don't think there's any magic for the integration, I imagine the data addon will just make a global variable (MountRarityData for example) and LiteMount (edit: RarityRandom function) will read it from there, no different from if it were packaged with LiteMount. The OptionalDependencies part of the ToC ensures that the data addon loads first.

I can't see any need to try to be fancy, it's probably literally just the Mounts.lua from your addon.

Good luck, I will await further developments with interest.

Thanks for thinking about and building out the skeleton.

I agree that loading the priority data from a second addon is better, as it separates logic from pure data.

In the coming days, I will have time for this. My plan is to build out the pipeline for automatic updates for my addon, and once that works, build out the data addon and its integration into LM.

I can see how you are thinking about integrating the two addons. I might need some advice on how to do that best. We'll probably discuss that in the PR, though.

Very excited for the opportunity and your openness to integrate the feature into LM! Thanks again for the bootstraping.

That's what I thought. I will push a PR and then we can talk about intricacies of WoW's Lua engine (if there are any).

Concerning the psychology, it's a very good point. The aforementioned Crab from the Trading Post reward came up in almost all tries when I was testing with my entire collection (around 300 mounts) because it was listed as 1% or something.
However, most of these extreme mounts are the original epic mounts, so it might not be the case for most people, or in this case be just a temporary effect.
Either way, I will test this out to see how it feels.

Last thing while I'm thinking of it, the 1/rarity as the weight isn't going to make anyone happy. There'll need to be some kind of mathematical operation to make them less extreme. For example if we look at these two mounts:

    ["6"] = 64.28086060062752,
    ["54"] = 0.004123711340206186,

here are the inverses normalized so "6" is weight 1:

   ["6"] = 1.0
   ["54" = 15588.108695652

Mount 54 is over 15,000 times more likely to be summoned than mount 6 which you will basically never see. In fact the small numbers of mounts that are very high weight (very rare) will appear to be summoned all the time.

While it is mathematically correct, pyschologically 1/r is not what people expect from "rarer mounts are summoned more often".

Edit: when you are thinking about this it can be helpful to start with the psychology rather than playing with math. Look at the rarities and ask yourself, if I mounted up 1000 times, roughly how many times would I want to get this rare mount versus this common mount.

Edit2: log(), log10() or sqrt() or even cube root[1] could be helpful transforms

[1] function root(n, r) return math.pow(r, math.log(n)/math.log(r)/r) end

So I updated my addon to use a new add-on I call "MountsRarity". The idea is that other addons could use this as well if they wanted to. There's really nothing special with the data in regards to LiteMount. However, should this addon be called something like "LibMountsRarityData" or what is the real convention in the WoW addon dev scene?
I will sort out the automatic releasing as time comes but I was already able to automate updating the data.

Additionally, I started work on the LiteMount changes.
My research was a bit disappointing in a few areas so in the interest of making my changes adhere to best practices and the way LiteMount works everywhere else, here are a few questions for you. Feel free to answer them when you have the time to.

1. The addon is loaded as an OptionalDependency. We should probably tell people about this option somewhere. Should that be in the README and/or somewhere else?
2. I wasn't able to find where the randomStyle on the context was actually set. I suppose that it is new and needs more integration? There probably needs to be a UI interface as well that allows the player to choose what style of randomization should be used.
   I would advocate for showing but disabling that option if the data addon is not present. This would make the functionality visible to new users.
3. If you know a better way to expose (and check for the presence of) the data, please let me know.
4. I also tried to understand see that the implementations of PriorityRandom and RarityRandom are quite similar. The first one looks at the "always" priority that you also mentioned before, so that is probably something that should also be included as well. Maybe we can unify the two to use a similar code path?

--

Regarding the psychology of the true distribution of the rarity values I found that it worked really good for my medium-sized mount collection. I would expect rare mounts to show up a few more times as they did and sometimes a common mount also got chosen.

It's really only for small collections where it feels weird. Specifically dragonriding mounts are by nature only a few mounts. LiteMount will always choose them on the Dragon Isles and both the Winding Slitherdrake and the Grotto Netherwing Drake are chosen very often. Sometimes 3–4 times in a row because only half the number of players own them compared to the Renewed Proto-Drake.

I will further look into your suggested math functions soon to fine tune the behavior.

The addon is now available at CurseForge as well: https://legacy.curseforge.com/wow/addons/mountsrarity (still waiting for manual review)

I assume this will be approved after the (US) weekend finishes, after which I'll probably put out the release.

I'm thinking about doing a reddit post about it. Since this is mostly your work, how do you prefer to be credited? As Sören Gade, or sgade, or a wow or reddit name?

I'm starting to feel like 1/n isn't prioritizing rare mounts enough. Maybe 1/(n^1.5) or 1/n^2 would be better. Not sure.

I'll let you know when the project should be publicly available on CurseForge.

On Reddit, you can credit me as SenseCe. 👍 Looking forward to your post!

How about you ask for feedback if rare mounts are not prioritized enough? Changing the weight in the random selection is pretty easy and straightforward to change.

@sgade This is pretty close to being ready to go, are you able to get MountsRarity onto CurseForge (and if you feel like it wowinterface and wago) so that I can point people at it if they want more-frequently-updated data?

Sure, I am using WoWUp myself which can handle GitHub as well but I'll take a look at how to get it published to the addon sites as well.

For better or worse, 99% of players use only CurseForge and maybe ElvUI directly. If it's not on CurseForge it doesn't exist.

The process is pretty simple.

1. Create the addon CurseForge then get the project ID from the description page under "About Project" on the right.
2. Add the X-Curse-Project-ID: line to your addon .toc
3. Create an API token on CurseForge https://legacy.curseforge.com/account/api-tokens
4. Add the CF_API_KEY: ${{ secrets.CF_API_KEY }} env: to your github action for Bigwigs packager
5. Add the CFI_API_KEY to github Settings > Secrets and Variables > Actions in either the repo or the org.

The bigwigs packager docs have details of all this I think.

I've added some texts for the rarity UI elements, if you have time to localize. Some are probably a bit complicated, I tried to be clear.

https://legacy.curseforge.com/wow/addons/litemount/localization/languages/94/phrases

Thanks for the instructions. Have a busy time right now as well but might get a few minutes in on the weekend for the account creation and deployment process.

I added a few translations but haven't looked at the UI in-game yet. Will do both and then give feedback once I think the two things are ready to go.

Came here to let you know that the add-on has been approved and should be available.

Happy to see 10.1.7-4 has been released :)

Yep I noticed that earlier, release is out!

Thanks for all your efforts, this is all your work and I am very appreciative.

I'll hopefully post something on reddit tomorrow when I'm not so tired and can string a few coherent sentences together.

You're welcome. I really appreciate your openness and the help you provided in getting it into LiteMount.

So happy that after many years of using your work, I can contribute back!

https://www.reddit.com/r/wow/comments/170cn3k/litemount_my_smart_mount_addon_for_retail_wow_now/

Hopefully it doesn't get deleted. I don't think it breaks any r/wow rules but who knows.

Awesome post. Thank you for the kind words.

Is there anything left to do for this ticket?

I was going to leave it open in case of needing to adjust the weights, but I think I'll open a separate issue for that.

Thanks again for all your work and making me look good. Please feel free to contribute more awesome features in the future. ;)