I tried to track down why my login/reload loading screen feels slow, here is a flame graph of CPU time used by addons: https://emmericp.github.io/Perfy/perfy-cpu-loadingscreen.svg (created using https://github.com/emmericp/Perfy)

Questie is an interesting case here because the reason why it's slow is somewhat unique: most of the time is spent loading and compiling localization files. Sure, almost all of the files execute as just immediately returning, but by the time they execute the time spent on compiling them is already lost.

I wonder if it would be worthwhile to put the translations into a huge string similar to how the database files are handled to avoid this problem?

Hey @emmericp

That is an interesting tool you built 👏🏻

To be honest: We don't have much time atm to look into big performance improvements, as Cataclysm is around the corner and there is much to do for us 😅

In the long run, it is totally worth it to investigate this further, as the load time of Questie can definitely be improved!

We were also thinking about moving the language code into sub-addons, so they can be loaded entirely on demand. But loading them from a big string, might be a good first step.

You are absolutely welcome to dig into this, if you want to.

So there's some interesting data in here.

So first off, Localization as a whole is not a concern, the real issue is Localization/lookups. Localization needs to always be available, as users can change Questie's language on the fly, without a reload. At 29.4ms I'm not concerned about that. Localization/lookups, on the other hand, refers to raw database overrides which are only used when Questie determines that it needs to recompile the DB. Localization lookup files are not used whatsoever during a normal session, so the fact that they're taking 294ms, or roughly a third of a second, per load screen is some pretty gnarly overhead for something that rarely gets used.

Digging in it looks like load time for localization files is, for the most part, directly proportional to number of lines. Looking at Localization/lookups/Classic/lookupItems/, if we compare ruRU to zhCN, going from the former to the latter is only an 8% increase in filesize, however it's a 150% increase in load time. If we compare the number of lines, it's roughly a 90% increase. This indicates there's something we should be doing differently, rather than the only optimization being compression.

Overall, the biggest offender is lookupQuests, taking roughly half of the time for all 'lookup' files. However, lookupNpcs and lookupItems, combined, make up the other half. What's interesting about that is, as far as I can tell, we don't even use lookupNpcs or lookupItems, because we get that data from the API. This is redundant data that we would normally only use on DB recompiles, aka each time the user changes languages or installs an update, and even then we don't even use this data because we get data from the API instead.

So to recap: half of the data we load with lookups (specifically Items and NPCs) is redundant, because we get that data from the API. We could simply not load this data and improve load performance by 40%. The other half of the data (quests and objects) is not redundant, but seemingly gets initialized every single time, regardless of current locale; if we could figure out a way to only load this data when we're actively needing to recompile the database with it, we could save another 40% of our load time in most situations.

Personally, I'm not too familiar with how l10n is structured - so unfortunately that last bit is going to probably be up to @BreakBB to figure out.

Feel free to dig into this again @emmericp - Your initial review was very helpful 🙏🏻

awesome, thank you!