Prerequisites

• [x ] I am running the latest alpha version of MineColonies and Structurize for my Minecraft version.
• [x ] I checked the MineColonies/Structurize wiki and made sure my issue is not covered there.
• [x ] I made sure that this issue is not a duplicate of any existing issue.

Context

• Minecraft Version: 36.0.43
• MineColonies Version: 0.13.660-BETA
• Structurize Version: 0.13.139-ALPHA

Problem description

In the log update of minecolonies, it says: Research datapacks must be placed into the minecolonies/researches directory, and can consist of :
research-jsons, which require the "id":stri...
But "minecolonies" folder does not exist (colonies does) and there is no datapack provided. Looks like it may have gotten cut off due to the length of the changelog content. I would have expected all datapacks to be located in the same /datapacks/ folder for consistency.

The same datapack contents also live under colonies/datapacks which seems redundant.

Notes or related things

If there is no datapack, then it'll be difficult for server owners to know exactly what is needed in them. If one will be provided, then it should also be listed in the main page of the mod with the other two links and comment stating which build it is required for (e.g. 0.13.660-BETA and higher). Thanks!

Viewers

• Add a thumbs-up to the issue to vote for it. This helps the request become more visible to the team and doesn't clutter the comments.
• Add a comment if you have any insights or background information that isn't already part of the conversation.

@gattsuru

Sorry, I've been working on getting some better documentation together for the wiki on data packs in general and the research datapack system in specific. Work and FRC support have just been slamming me.

By default, user or modpack-created datapacks must be stored in the minecraft/saves/<worldname>/datapacks directory. See the vanilla wiki for the general version on this. There are some mods out there that will automatically copy them from some other directory at the instance level into new worlds. I'm not sure if that's how the colonies-directory stuff happened, as it shouldn't be occurring from a vanilla+structurize+minecolonies stack. They can be stored as zip files, or as unzipped directories -- I find the latter easier if you're modifying one.

Data packs can also be added within mods themselves, which is how the default Minecolonies research (and sifter drops, and advancements, and crafter recipes, and block/item tags) works. This avoids possible versioning or complexity-of-installation problems. You can see the current contents of the research directory here, or simply unzip your minecolonies-jar file locally (preferably in some directory other than your mod instance). It's still possible to override (by including files in filesystem-level datapacks with the same name) or remove (by using the remove property) mod-created researches.

I've uploaded a simple sample datapack, which locks the Warehouse building behind a new branch of research that has reduced research times, for a rough idea of how the system can work (though it's still very basic; there are some more advanced concepts like research-requirements, autostart or hidden research, so on, that are a little too complex to add here). Should give you a rough idea of what the system can do.
SampleResearchDataPack.zip

Will update you once the wiki documentation is together enough to be published.

This update is in the latest alpha but NOT in the latest beta yet. Are you aware of that?

Ray this is 660 beta which is the absolute latest on curse

Huh, who launched 660

i think orion? not sure

@gattsuru you can close this issue if after its solved if necessary.

Thanks for the quick response on this. :-)
The datapack folder is CORRECT on the client-side, but not on the server-side which is where I'm reporting the issue for. DataPacks can exist in one or both locations, but typically for MP Servers, they exist in both.

You are 100% correct about the .zip file and most (IMO) never package the datapacks correctly and often missing key files or include other files that don't belong. I've been trying to educate developers of datapacks since they came out in 2018 more often to understand how to package them properly and to provide clear instructions for CLIENT & SERVER installation folders. Sites like files in a jar or zip to make distribution easier, but users need to be reminded that DataPacks are not the same as ResourcePacks (which are zipped files).
I'm of course happy to help any way I can since I tend to stay on the latest versions of this mod including testing if I know what I'm testing for. -Steve

do you actually want something to do for mcol (possibly testing)? if so hit me up on our discord :D

If I could get the initial DP, I'd be happy to give it some testing. Both my wife and I play daily and create multiple colonies in various biomes. Our current one is on the Nether Roof - on top of the Nether.

Data packs can't exist on a solely client-connecting-to-a-server situation: basic Minecraft + Forge doesn't actually include a way to even select them on a remote client, since they're tied to a specific world save. I've implemented a data transfer system to ship the data from the server to the client on connect (and on datapack reload), to resolve the problem.

Are you having problems with that system, in the sense of remote clients not seeing researches in the University? Or do you want a copy of the current data pack to have a base to modify? Or something else I'm missing?

((I'd prefer to avoid leaving loose copies of the jar-internal datapack around; not only is it a bit of extra disk space, it's asking for people to copy the whole thing when tweaking just a couple files, and there are some potential unexpected behaviors if multiple zip- or folder-style datapacks do that.))

@Raycoms @ravenbuilder934 Beta builds are now triggered every sunday 12:00 PM GMT+1.

This should keep Betas relatively up to date.

DefaultMinecoloniesResearchDataPack.zip
This copy is current to March 1, 2021, and includes just research (loot_tables, tags, and crafting recipes are very likely to change w/ short notice). Modpack makers should generally not include full copies of this sorta thing; normally, you'd only want to include those researches or effects that are being altered or added.

Thanks @gattsuru - I am a developer, server owner (I host our own servers) and create modpacks so that our users can get the latest updates for our servers. I know the wiki page needs more details around this feature, but for my knowledge how will I know when to add to the datapack new researches or effects being altered or added? Seems like a gap in what the server owner has to do (vs single players), to ensure only the changes we want get included. Thanks! -Steve

Normally, a server owner will not need the external zip-style data pack at all: these default researches are included inside the mod jar itself, which is automatically loaded for all worlds, updated with jar updates, and is transferred to connecting remote clients. If they're adding a modpack or privately-created datapack, only then do you need to add a file to the world/data directory, and it only needs the minimum files to describe the changed or added research, and this only needs to be installed on the server side, as clients (intentionally, under Forge/vanilla rules) have no control over what datapacks are applied to a remote world.

I've tested this behavior both when connecting to a server on the same computer, a server across a LAN connection, and across the internet: if you're seeing problems, I'd appreciate specific details regarding your situation and the behaviors you're seeing.

The server owner should only need to include the exact files that they're intending to modify (or create, or remove) in a zip-style data pack; adding external data packs only adds to or overrides (and can not turn off) jar-internal data packs. If additional research is added into a new minecolonies jar, it'll simply merge alongside those zip-level changes. The only potential version conflicts are if a new research, effect, or remove-research is added into the jar with the same ResourceLocation as a zip-added one (unlikely), if someone removes a research which has descendant researches added, or if they've modified a base research or effect which was later changed in minecolonies itself. The latter won't happen often (and when it does, is likely to be small stuff like tweaking costs, vertical alignment on the research grid, requirements, or hidden status), and in which case, the user-modified version will take precedence... which is the intended behavior, as players are quite likely to want their versions to 'stick'.

Not usable because the min and max versions are wrong . I'm surprised no one caught this before posting on curse .

Not usable because the min and max versions are wrong . I'm surprised no one caught this before posting on curse .

That's an entirely different issue. It actually isn't an issue at all tho, that's completely intentional. However we just released 662 alpha which accepts 143 of structurize