Inquiry regarding relative lack of optimization with configs
asiekierka opened this issue · 5 comments
A recent heap dump analysis of Enigmatica 2 Expert made me notice that this mod creates a 112k-line 4MB "blocks.cfg" file.
Now, this is understandable given the purpose of the file. However:
- this leads to a 25MB object (for very large modpacks) being constantly kept in memory here,
- due to the large filesizes and regeneration on every load, this could lead to unwanted I/O usage.
In order, I would propose fixing this in the following manner:
- It appears the (at least non-"overwrites") config objects are not used after postInit, so I think they could be safely set to null after saving. This will fix the memory usage problem,
- Ideally, I would move the config file generation to an admin-only chat command of some kind, as opposed to every boot. This way, the files are only generated on demand, saving loading times, disk space, etc. - and, as I understand it, only the overwrite configuration files are used anyway? (If that's not possible due to modpack makers' dislike for such a system, perhaps the behaviour of generation on every boot could be predicated upon a config option?)
I'm willing to help out with the implementation part of the process, conditioned on the mod being released under a FOSS-compliant license.
- It appears the (at least non-"overwrites") config objects are not used after postInit, so I think they could be safely set to null after saving. This will fix the memory usage problem,
Yeah, I can do that.
- Ideally, I would move the config file generation to an admin-only chat command of some kind, as opposed to every boot. This way, the files are only generated on demand, saving loading times, disk space, etc. - and, as I understand it, only the overwrite configuration files are used anyway?
That is not possible, as I apply some modifications during the startup and at the point, an admin could use chat commands, the original state would be lost. Though, a config option should work though.
That's fine. I just don't want to waste drive I/O - saving a 4MB file takes some time, and can probably cause SSD wear with a modpack rebooted way too many times (though probably not a real concern in practice).
All I'm worried about is that modpack developers forget to disable the option for production. Immibis would solve this by making the option a toggle (if you set it to true, it fires once, then is set back to false and the config file is re-saved), but modpack makers might not like that either. Up to you.
The default will be not to generate the files, as you said, only mod pack devs would be interested anyway.
Fixed in 1.6.9. Will be up soon™