Addon memory usage
silverwind opened this issue · 30 comments
I'm seeing around 70KiB/s memory growth from Questie in PerformanceFu shortly after login which causes the LUA garbage collection to run approximately every 3 minutes as opposed to every 50 minutes with the AddOn disabled (GC causes a short freeze of the game).
My script memory setting is currently at 512MB, but it appears to have no influence on the GC threshold. Could it be that there are memory leaks in the addon which cause the unusual high growth?
Without Questie:
With Questie
Not sure which AddOn's you're running but here is mine after spending a couple hours going through some of the OnUpdate functions and moving a few things around.
Before Tweaks:
Average: 72.1 KiB/s
After Tweaks:
Average: 22.2 KiB/s
Without Questie:
With Questie:
Questie is a memory pig there is no doubt there. But this is the first time in months someone has complained about lag or poor performance. Hopefully my tweaks will work for you. I'll get my changes committed but you'd have to manually install the build until it makes it into an official release.
I installed the latest release so this is WITHOUT my new tweaks just for an unbiased 1:1 comparative baseline. I think the numbers speak for themselves.
I double checked after disabling Questie and the numbers were virtually the same as my original screenshot in the above post.
With Questie:
Looking good. The issue came to my attention because I've had garbage collections happening every 30s which was rather annoying. I've found that CustomNameplates (up to 400kb/s), FastQuest (up to 200 kb/s) were even worse offenders than Questie. 30kb/s is totally acceptable. I will test your changes later today. Thanks!
Those addons do a lot of OnUpdate, meaning code is being called on every single frame (and they probably don't cache anything). For Nameplates, there is no other possibility, because they are drawn on WorldFrame withe very rendered frame. Questie is, considering its size, pretty performant.
Well done on the changes, Dyaxler.
That's still a pretty damn good improvement over the release build. Even the one I checked in last night didn't increase much. I did however find out what is taking up so much memory. The QuestieTracker:fillTrackingFrame() - its constantly calculating distance for all quests in the list and shuffling them in order from closest to furthest. Slowing it down to once every 5 seconds gave the largest performance boost. When I have a huge chunk of time I'm going to see if I can't streamline that whole function; break it apart so I have better control over what it's doing. For example, it only needs to fill the tracker with objectives when a quest is added or removed from the quest log tracker or when a quest updates. That can be triggered by events. But the distance calcs should be done faster - not in real time. Once every 10 seconds should be more than enough for the tracker sorting. However, the QuestieArrow depends on this function as well to get the closest objective's distance it's pointing at - there can be more than one point for a quest or steps that the arrow switches to. That data is being cached (formally at 60 times a second) now slowed down to once every 5 seconds. The arrow and tracker list will update but when a character is running they will hardly notice the lag. I don't hardly notice it. Other things also trigger the QuestieTracker:fillTrackingFrame() but at least the OnUpdate function has been slowed down.
So yeah... it'll take me some time.
By the way... if you're worried about performance don't use the QuestieArrow. Holy cow that thing is a pig. You'll see a 120% increase in memory usage once it shows up on the screen. No way to slow that down because that is calc direction and distance based on your clients framerate. So if you're pumping 60+ FPS then it's calc 60 times a second. This is to keep the arrow smooth while moving around and giving an accurate distance calc. I removed the Time To Arrive calc - kind of pointless and it reduced the foot print by approximately 24%. I tried slowing it down slightly by dividing the frame rate by 1/2 but then the arrow looked really horrible when moving around. I'd rather take the mem hit to keep it smooth.
Yeah, I see that my framerate is part of the reason why it's so high. I run uncapped at 200+ fps because VSync just doesn't seem as smooth. By the way: after 2 hours of gaming memory went down to 60k-100k, which is totally fine. Oh, and I'm not using the arrow.
My monitor sucks and is only capable of 60 Hz so I have my FPS capped at 60 FPS... it's an older 27" Ultrawide screen so I'm running in a Maximized Window @ 2056 x 1080 so I can use my other three monitors to update code, chat on Discord etc. The game runs smoothly with no lagging at anytime so I was a bit surprised when you said you were experiencing some lag. But after looking at and narrowing down which functions were causing the excessive usage... I'm surprised it's not a bigger issue for some folks.
My inner obsessive compulsiveness is screaming to make that function more efficient now that I know it's gobbling up a ton of memory. I know I can make it better. :)
@silverwind I put in some more work tonight. I'm getting closer.
With Questie:
Without Questie:
So according to this Questie is only consuming 43.8 KiB/s
Interesting! Here is with Just EQL3 - FuBar and PerformanceFu loaded:
With Questie:
Without Questie:
I wonder what mod I'm running is gobbling up all my memory cycles... hmmm....
Few results on the latest commits:
ae2418c - 52 KiB/s
71eb4c6 - 55 KiB/s
805fc60 - 49 KiB/s
04578d2 -110 KiB/s
The latest tweaks seem to only have a minor effect on my side. I had the CorpseArrow turned off in all tests. Could it be that some CorpseArrow OnUpdate code is still active even when it's disabled?
Which latest commits? I didn't commit anything yet. :) The screen shots from last night is still beta code on my test box. :P
Ah, sorry. I guess I confused 9d1dfc4 with being your latest tweaks 😉 Awaiting your commit.
Kind of a shame we don't have memory/cpu profiling available in the 1.12 client (they came in 2.1) which would make this stuff a lot easier.
Please try the new release and let me know what you think. I have a feeling you might be surprised.
https://github.com/AeroScripts/QuestieDev/releases/latest
Woah, just 11 KiB/s with 2ea7660 (6 KiB/s with Questie disabled, 90 KiB/s with 2 commits earlier).
Awesome work. This is definitely fixed. 👍
Yeah, I didn't even bother with posting screenshots. I wanted the surprise to be... well... a surprise. :)
I even re-added some old code that caused an even larger memory issue several months ago. If you enable the background (/questie background) it'll auto size the background up - down - left - right and frame your tracked quests perfectly. Looks bloody awesome.
I even fixed a super old issue with Quest drops not registering on the tracker upon first drop. In the past it would update after picking up the second drop. Sometimes it would happen with "kill x number of mobs". The first kill wouldn't register until the second etc...
/questie background
Looks quite sexy, I'll keep it that way.
Quest drops not registering
Nice, I noticed this too. My first thought was that the update frequency was throttled. Great to see that fixed too.
Nope. It was triggering too many calculations and because of the way the functions were rebuilt and throttled, it would only update once every 5 seconds making it useless.
If I find stable TomTom like arrow code that I can back port to Vanilla without it causing HUGE amounts of memory issues, I'm sure it'll have a Time To Arrival feature.
It's a patch... I know what's going on. I was being lazy and wasn't checking to see how many buttons were being added before the resize tracker function call. Each tracked quest is actually being created separately as a button. This allows us to dynamically change their order from closest to furthest. Same behavior gets applied to the quest objectives when you have an arrow attached to it. I just need to move the margin modifiers from buttons to the tracker frame so it doesn't get added into the buttons size. There will be less calculations once I get it working right.
I just need to create a "finished creating buttons" flag in the fill tracker function. Once done THEN it fires the resize tracker function.
This allows us to dynamically change their order from closest to furthes
Neat, didn't know it does that. Would it be possible for the arrow to then always track the topmost (nearest) objective? It's a feature I remember from TomTomLite 😉. Maybe as some sort of autoarrow option.
I was just thinking about that last night too... that's going to be a totally new feature. I'll have to think on it's logic flow before I start injecting smarter behavior like that. :P
The QuestieArrow code is till REALLY messy and it's calcs are expensive. I'm going to see if I can't find a decent version of TomTom or some other similar AddOn from BC or Wrath and see if I can't back-port it. That way I don't have to write my own algorithm for drawing the arrow.
The corpse arrow should take priority when dead. Otherwise, track the nearest objective of all tracked quests. :)
...I have the update function for that throttled to once every 5 seconds. So, this should work and only update once every 5 seconds. Even if a player is hanging out at the mid point of two objectives and keeps bouncing back and forth. It shouldn't drive players nuts to see their arrow jump back and forth as much as it would be if this function were running at 30x-120x per second. This was the old rate. It was based on the users detected FPS...
I should build in a toggle though... some users are going to want full control over what their arrow is pointing at.
Looking at TomTomLite, the forced updates happen in 2 seconds granularity, which I guess is acceptable, but a config option wouldn't hurt, yes.
corpseArrow = true,
forceUpdateThrottle = 2.0,
goodcolor = {0, 1, 0},
badcolor = {1, 0, 0},
middlecolor = {1, 1, 0},
