Permissions / Group being deleted without user / api / command input.
MacTh3Mac opened this issue ยท 5 comments
Description
This report is in follow up to Issues #2764 & #2751
Reproduction steps
Assign "temporary" permission to user
Wait indefinite amount of time
Assign permanent permissions/groups to that user
Wait indefinite amount of time
"Temporary" permission expires (or is removed via command) and any perms added after that "temporary" node are removed
NOTE: This does NOT occur all the time, but is more frequent than before the last fix by luck in PR #2767
Expected behaviour
User does not lose permanent groups / permissions
Environment details
Servers mixed 1.12.2 PaperSpigot / latest and 1.16.4 Paperspigot latest
Luckperms 5.2.61
DataBase MySQL 5.7.32
Messaging - Redis
620,000 approx records in _players table
850,000 approx records in _user_permissions table
DB Server on 10GB LAN with servers and running at low cpu usage.
Any other relevant details
Since the possible fix in PR #2767 by Luck, this issue has not been resolved. Anecdotally the issue has gone from being an occasional annoyance to a daily occurrence.
Suggested Fix
Don't ever write back deletion to MySQL table. The database must always be the ultimate source of known good data.
If a plugin / api call or command requests the removal of a permission then remove that permission, There should never be a situation when the databases version should be considered invalid as all permissions are written out accurately when applied.
Next Steps
Happy to provide detailed logging of future incidents if required, but ultimately the logs from prior 2 x issues should cover the problem.
Interesting that the previous change has made this worse... I'm not sure what conclusion to draw from that.
Suggested Fix
Don't ever write back deletion to MySQL table. The database must always be the ultimate source of known good data.
If a plugin / api call or command requests the removal of a permission then remove that permission, There should never be a situation when the databases version should be considered invalid as all permissions are written out accurately when applied.
This is exactly the fix I applied - when the temporary permission expires, a single query to remove that permission only is sent to the DB. All other records are untouched. You should be able to quite easily confirm that this is the case by monitoring the SQL queries being executed.
Do you have any plugins which interact directly with the LuckPerms API that could be either causing or contributing towards this issue?
Have you updated all servers in your network to the new version? (including any BungeeCord proxies, etc)
I will set up enhanced logging and advise shortly what queries are causing this. Please give us a few days.
After further testing with Latest release on all proxies, and backend servers we have seen no reoccurrence in 7 days.
We found a rogue V4 instance running on the network.
Thanks for the fix Luck.