Split large single save file into multiple smaller files
blablubbabc opened this issue ยท 2 comments
Currently whenever the save file gets written because of either a single shopkeeper was modified or some shopkeepers were modified in the last 5 minutes (depends on config settings), ALL shopkeepers need to be saved in order to write the save file. The serialization of the shopkeeper data needs to happen on the main thread (only the raw file operations are getting handled on a separate thread).
This might potentially result in a notable overhead if the total number of shopkeepers (and the amount of their data) is very(!) large. For most servers this shouldn't be an issue, because the number of shopkeepers would have to be quite large before this becomes a notable problem. (Saving 10000 empty shops requires ~15ms on the main thread, TODO: Actually test this properly for non-empty shops)
But nevertheless it would be nice to improve this, especially as this seems to be rather easy to implement.
Also this might reduce the risk of total data loss in case there happens to be some issues when saving a shopkeeper.
Maybe shopkeepers could be grouped by either world or even by chunk. Saving each shopkeeper individually in their own file seems overkill to me right now, and might result in even more total overhead when considering file operations and wasted disk space.
TODO: Ask large servers for the size of their save files (and total number of shopkeepers), the saving overhead they observe (logged in the console), and information on how their shopkeepers are distributed across worlds.
If anyone happens to come across this ticket and have real-world data on this, let me know.
Server-Version: git-Paper-125 (MC: 1.14.3)
Shopkeepers-Version: v2.7.2
HDD-Type: SSD
Since MC 1.14 i get a slight delay when opening a shopkeeper trading-window.
save.yml: https://pastebin.com/GnM6Rr2J
config.yml: https://pastebin.com/hz2F6qmX
Since MC 1.14 i get a slight delay when opening a shopkeeper trading-window.
Yes, I observe that as well when I open any custom / plugin created window for the first time (afterwards it seems to no longer occur). I even get this on a local server. Since it affects other plugins as well, my guess is that this is probably some minecraft or Spigot issue. I have no idea if and how it would be possible for plugins to avoid that.
I don't think this is related to this issue.
Edit: I created a separate ticket for this. I think I was also able to find a workaround.