[REQUEST] bulk cleanup customer history

Question

[REQUEST] bulk cleanup customer history

zandadoum opened this issue 10 months ago · 28 comments

zandadoum commented 10 months ago

Hello,

I'd like to ask for a couple options to bulk clean customer history with 3 options:

by days: bulk delete every customer history that didnt have activity/messages for past X days (if timestamp wasnt recorded in previsou version, maybe implement it for the future)
by tip: bulk delete every customer history that tipped less than X gold
a combination of 1+2: bulk delete every customer history older than X days that tipped less than Y

thank you!

derfloh205 · Answer 1 · 2023-11-30T12:03:28.000Z

@clery
Since you introduced the module do you want to work on that? otherwise I'll take a look at some point

clery · Answer 2 · 2023-11-30T13:20:35.000Z

Hey @derfloh205, sorry these days I really lack the time 😕 But it is an interesting feature 😄

derfloh205 · Answer 3 · 2023-12-01T07:58:26.000Z

Hey @derfloh205, sorry these days I really lack the time 😕 But it is an interesting feature 😄

No problem I feel you! :D
Then Ill look at this at some day

derfloh205 · Answer 4 · 2023-12-18T12:01:28.000Z

12.0.0 will introduce a bulk delete for all customers with 0 tip, more options may come in future updates :)

derfloh205 · Answer 5 · 2023-12-14T14:26:07.000Z

@clery I have a question regarding the customer history. Where does ace save the db to ? the saved variable seems to be different than what can be found in the .db property of the ace addon

clery · Answer 6 · 2023-12-14T15:29:41.000Z

From what I can see and recall, it’s in WTF/Account/AccountName/SavedVariables/CraftSim.lua
Within this file it’s in a variable called CraftSimCustomerHistory

Hope it helps !

derfloh205 · Answer 7 · 2023-12-15T05:07:01.000Z

@clery
ah ok, I thought of that but could not find anything ingame when /dump or /tinspect on it hmm
I just need to know the structure of it so I can migrate a player's data to my new structure

Do you happen to have some data you can share?
Otherwise I can also just test with alts

clery · Answer 8 · 2023-12-15T09:18:37.000Z

Yeah so I was a bit of an idiot. I was comparing strings for version comparison in the migration code (which obviously couldn't work). I suppose it could've created some issues since the 10.0.0+ version of CraftSim !
Hope it helps

derfloh205 · Answer 9 · 2023-12-15T06:49:33.000Z

@clery also it seems your migration function always cleared the saved variable 🤔 I cant really seem to understand the code there

clery · Answer 10 · 2023-12-15T08:07:03.000Z

Hey @derfloh205
So here's an example of what data may look like

CraftSimCustomerHistory = {
	["profileKeys"] = {
		["Rakhnar - Dalaran"] = "Default",
	},
	["realm"] = {
		["Dalaran"] = {
			["lastCustomer"] = "Clientname-Servername",
			["Clientname-Servername"] = {
				["totalTip"] = 70000000,
				["history"] = {
					{
						["to"] = "Message to customer 1",
						["timestamp"] = 1684509157045,
					}, -- [1]
					{
						["from"] = "Answer from client 1",
						["timestamp"] = 1684509209388,
					}, -- [2]
					{
						["to"] = "Message to customer 2",
						["timestamp"] = 1684509223438,
					}, -- [3]
					{
						["crafted"] = "|cffa335ee|Hitem:193000::::::::70:260::13:8:8836:8840:8902:8780:9405:8791:9373:9366:8:28:2164:29:49:30:32:38:8:40:259:49:194578:50:204682:51:204440:::::|h[Ring-Bound Hourglass |A:Professions-ChatIcon-Quality-Tier5:17:17::1|a]|h|r",
						["timestamp"] = 1684509301251,
						["commission"] = 70000000,
						["reagents"] = {
							{
								["isBasicReagent"] = true,
								["reagent"] = {
									["quantity"] = 2,
									["itemID"] = 192836,
									["dataSlotIndex"] = 1,
								},
								["source"] = 0,
								["reagentSlot"] = 1,
							}, -- [1]
							{
								["isBasicReagent"] = true,
								["reagent"] = {
									["quantity"] = 1,
									["itemID"] = 192871,
									["dataSlotIndex"] = 2,
								},
								["source"] = 0,
								["reagentSlot"] = 2,
							}, -- [2]
							{
								["isBasicReagent"] = true,
								["reagent"] = {
									["quantity"] = 1,
									["itemID"] = 193379,
									["dataSlotIndex"] = 3,
								},
								["source"] = 0,
								["reagentSlot"] = 3,
							}, -- [3]
							{
								["isBasicReagent"] = true,
								["reagent"] = {
									["quantity"] = 3,
									["itemID"] = 193370,
									["dataSlotIndex"] = 4,
								},
								["source"] = 0,
								["reagentSlot"] = 4,
							}, -- [4]
							{
								["isBasicReagent"] = false,
								["reagent"] = {
									["quantity"] = 1,
									["itemID"] = 194578,
									["dataSlotIndex"] = 7,
								},
								["source"] = 0,
								["reagentSlot"] = 7,
							}, -- [5]
							{
								["isBasicReagent"] = false,
								["reagent"] = {
									["quantity"] = 1,
									["itemID"] = 204682,
									["dataSlotIndex"] = 8,
								},
								["source"] = 1,
								["reagentSlot"] = 8,
							}, -- [6]
							{
								["isBasicReagent"] = false,
								["reagent"] = {
									["quantity"] = 1,
									["itemID"] = 204440,
									["dataSlotIndex"] = 9,
								},
								["source"] = 1,
								["reagentSlot"] = 9,
							}, -- [7]
						},
					}, -- [4]
					{
						["to"] = "Message to customer 3",
						["timestamp"] = 1684509305521,
					}, -- [5]
					{
						["from"] = "Answer from customer 2",
						["timestamp"] = 1684509316697,
					}, -- [6]
				},
			},
		},
	},
}

Weird this migration clearing db 🤔 I've updated CraftSim regularly since then and haven't got any data loss.
But there is indeed something weird, there should be a "version" field as well in there, which is missing, and could lead to migrations running everytime... not sure why though

derfloh205 · Answer 11 · 2023-12-15T09:35:09.000Z

Yeah so I was a bit of an idiot. I was comparing strings for version comparison in the migration code (which obviously couldn't work). I suppose it could've created some issues since the 10.0.0+ version of CraftSim ! Hope it helps

Yeah I found he culprit by testing ^^

I guess that means that all data has been cleared for all players that used it after 10.0.0?
Its bad yes but on the other hand.... I dont need to migrate any data to my version haha

derfloh205 · Answer 12 · 2023-12-15T09:36:50.000Z

Yeah so I was a bit of an idiot. I was comparing strings for version comparison in the migration code (which obviously couldn't work). I suppose it could've created some issues since the 10.0.0+ version of CraftSim ! Hope it helps

Its tricky tho, it sometimes works like 9.0.0 is lower than 10.0.0
but 7.10.9 will be greater than 9.9.9 xD
very wierd and a bit of a trap

clery · Answer 13 · 2023-12-15T09:39:08.000Z

I guess that means that all data has been cleared for all players that used it after 10.0.0? Its bad yes but on the other hand.... I dont need to migrate any data to my version haha

Actually I don’t think so, this morning as I was testing with 11.x.x yes there were some weird behaviors but I didn’t lose anything !
But hey you could still do a purge xD

derfloh205 · Answer 14 · 2023-12-15T09:42:23.000Z

I guess that means that all data has been cleared for all players that used it after 10.0.0? Its bad yes but on the other hand.... I dont need to migrate any data to my version haha

Actually I don’t think so, this morning as I was testing with 11.x.x yes there were some weird behaviors but I didn’t lose anything ! But hey you could still do a purge xD

hm thats even wierder, why is it gone for me tho ^^ and never saves

derfloh205 · Answer 15 · 2023-12-15T09:47:53.000Z

I guess that means that all data has been cleared for all players that used it after 10.0.0? Its bad yes but on the other hand.... I dont need to migrate any data to my version haha

Actually I don’t think so, this morning as I was testing with 11.x.x yes there were some weird behaviors but I didn’t lose anything ! But hey you could still do a purge xD

So you still have a lot of data?
Could you do me a favor? I would give you my current craftsim version with the new datastructure and the migration
in game you just run
/tinspect CraftSimCustomerHistoryV2 and check if the data makes sense ^^

derfloh205 · Answer 16 · 2023-12-15T09:52:01.000Z

its pushed ^^ the newest commit on main
It does not delete any old data (yet ^^)

clery · Answer 17 · 2023-12-15T10:27:52.000Z

Sure I’ll try that asap

clery · Answer 18 · 2023-12-15T10:49:12.000Z

So, from what I can see the SaveVariable was not removed, but it looks like DB is not loading properly and looks empty 😮

derfloh205 · Answer 19 · 2023-12-15T10:53:22.000Z

So, from what I can see the SaveVariable was not removed, but it looks like DB is not loading properly and looks empty 😮

hmm can you
/run CraftSimOptions.CustomerHistoryMigrationDone=false
and try again while looking at the debug for CUSTOMER_HISTORY?

derfloh205 · Answer 20 · 2023-12-15T11:02:22.000Z

ah thats a new method in GUTIL
do you have it local as a submodule?
https://github.com/derfloh205/GUTIL

clery · Answer 21 · 2023-12-15T11:01:14.000Z

Here's what I got

CustomerHistoryLegacy Migration failed:
...rface/AddOns/CraftSim/Modules/CustomerHistory/DB.lua:87: attempt to call method 'TrimTable' (a nil value)

Are you still using AceDB ? I can't seem do find any reference to it

clery · Answer 22 · 2023-12-15T11:19:21.000Z

Although I still can't seem to look at the history from the Custom History frame, but I'm guessing it's still WIP 😄

clery · Answer 23 · 2023-12-15T11:18:13.000Z

Oh right, I missed an error when I pulled.

Well it looks great ! Did some basic tests, send myself messages, completing a crafting order, data looks nice and complete 😄

clery · Answer 24 · 2023-12-15T11:21:31.000Z

Not sure if that's normal though

The item link looks empty ? But it may just be an issue with the inspector

derfloh205 · Answer 25 · 2023-12-15T11:23:47.000Z

Not sure if that's normal though The item link looks empty ? But it may just be an issue with the inspector

yes I also wondered xD thats an inspector thing, the itemlink is there!
the UI is WIP :D it was just about the data migration so that your old data is converted into new without issue :)

thank you so much for testing <3

clery · Answer 26 · 2023-12-15T11:28:10.000Z

It’s my pleasure ! Thanks for improving the customer history man ❤️

derfloh205 · Answer 27 · 2023-12-15T11:29:49.000Z

It’s my pleasure ! Thanks for improving the customer history man ❤️

Thank YOU for initially creating it :D

derfloh205 · Answer 28 · 2023-12-15T11:30:00.000Z

wups misclicked

Share to