Problem: MC took a very long time-- hours-- to load with this mod installed. Much of the time was spent with this message:
Supplied images for extraction contained few differing colors; attempting clustering color extraction.

I'm not sure if this is a one time thing to generate assets but even once it takes too long. I suspect its due to added mods, but I believe part of the point of this mod is to add ores to mod-added blocks.

Here is my boot log from MultiMC:
https://paste.ee/p/tb7k2

Ver of DAG 1.1.0
Excavated Variants 1.0.3

Hmm, that line usually doesn't get called that much. Have you tried:

• Deleting the globalresources folder and letting it regenerate?
• Removing Oculus and Rubidium (worth a shot)?

If that doesn't work, turn on asset caching in the DynAssetGen folder, and then share a .zip of the generated asset caches

Additionally, do you have any custom configs or resource packs?

Yeah, I'll need a bit more info about configs on your end, as I tried to reproduce this and failed: https://gist.github.com/lukebemish/b005282ff181f5cb234f2c3b65445cc0

I'll provide whatever you need. As far as resource packs, I do have some. Not sure if this will work, but I'm pasting a screenshot of my resource pack folder. The disabled ones are not being used.

As for configs, I wiped the entire config folder before launching.

My global resources folder only has the Excavated Variants zip.

I just tried without Rubidium and Oculus but still the same result. When I remove DAG and Excav Variants it flies through like normal.

If I remember, I'll do the asset caching later, as letting it run through will be an all-night affair. Hopefully that other user will post a log or something to help you find some correlation.

Hmm. Did you try deleting the globalresources folder of just the configs? Could you try deleting both (at the same time; this matters), make sure every mod is updated to the newest version, and try again? I'm still unable to reproduce this on my end with the same mod list

@EldrinFal, I've done some asking around since I've been unable to figure this out much on my end (and I likely won't be able to super easily). Can you try a couple of other things for me?

• Remove all your resource packs and try again
• Remove geckolib and anything that depends on it, and try again (this one may be a long shot, but still worth a shot since I can't reproduce this)

I have no clue what that could be, and it doesn't sound related to DynAssetGen. Are you experiencing the issue mentioned in this issue? If not, please open a new one with relevant information. If so, please provide more details.

it mentions your mod infront of the warnings about the unsupported metadata
i will do make a new issue if anything is becoming incompat with newer forge version in 1.18.2

Sure; that's just a warning; it doesn't mean anything. Once again, are you experiencing the issue described in this issue (that is, the loading time stuff)? If so, provide full logs and more details. Otherwise, open a new issue

Hmm. Did you try deleting the globalresources folder of just the configs? Could you try deleting both (at the same time; this matters), make sure every mod is updated to the newest version, and try again? I'm still unable to reproduce this on my end with the same mod list

what happens if your instance if the mod is not generating a globalresources folder at all ?

Hmm. Did you try deleting the globalresources folder of just the configs? Could you try deleting both (at the same time; this matters), make sure every mod is updated to the newest version, and try again? I'm still unable to reproduce this on my end with the same mod list

what happens if your instance if the mod is not generating a globalresources folder at all ?

What version are you on?

Hmm. Did you try deleting the globalresources folder of just the configs? Could you try deleting both (at the same time; this matters), make sure every mod is updated to the newest version, and try again? I'm still unable to reproduce this on my end with the same mod list

what happens if your instance if the mod is not generating a globalresources folder at all ?

What version are you on?

im on 1.18.2 of the mod
dynamic_asset_generator-forge-1.18.2-0.6.3.jar

1.18.2 is not actively developed, and does not use the defaultresources system, so that makes sense.

1.18.2 is not actively developed, and does not use the defaultresources system, so that makes sense.

i was getting a warning about metadata language and refinedstorage is not supported in my logs

I have no clue what that could be, and it doesn't sound related to DynAssetGen. Are you experiencing the issue mentioned in this issue? If not, please open a new one with relevant information. If so, please provide more details.

@EldrinFal, I've done some asking around since I've been unable to figure this out much on my end (and I likely won't be able to super easily). Can you try a couple of other things for me?

• Remove all your resource packs and try again
• Remove geckolib and anything that depends on it, and try again (this one may be a long shot, but still worth a shot since I can't reproduce this)

the metadata issue only happens before forge 1.18.2 -80 doesn't appear after build 79
but loading older worlds has become oddly weird after having to update forge past forge build 69
so what ever happen in 1.18.2-40.1.69 after that is breaking things in mods that haven't been updated since that time area

The metadata stuff is just a warning and can be ignored

@SDUBZ, are you actually having an issue? If so, please describe it. So far, you've just described warnings and whatnot. Please, once again, either add more details if it's the same issue described originally here or open a new issue if it's not

@SDUBZ, are you actually having an issue? If so, please describe it. So far, you've just described warnings and whatnot. Please, once again, either add more details if it's the same issue described originally here or open a new issue if it's not

my worlds have always taken more then 15 mins in 1.18.2

Alright, that's a different issue. This one's about game loading. Make a new issue, give me logs, all that stuff.

Alright, that's a different issue. This one's about game loading. Make a new issue, give me logs, all that stuff.

i looked at this guys logs and it really seems like his forge or fabric is really messed up
missing sounds and more even the native mc text to speech is broken
also hes missing the turtle api for computer crat

Alright, that's a different issue. This one's about game loading. Make a new issue, give me logs, all that stuff.

i looked at this guys logs and it really seems like his forge or fabric is really messed up missing sounds and more even the native mc text to speech is broken also hes missing the turtle api for computer crat

Those are just standard log warnings with that set of mods. They happened when I tried to reproduce this, so they're likely irrelevant to the issue

@EldrinFal Okay, I can reproduce this with that"128x earthy texture pack". Can you see if removing that pack fixes it? I'm looking into why that causes issues now.

@SDUBZ, can you open a new issue with your information? I'd like to look at it but that really needs its own issue.

After a bit of profiling, it seems like something in that resource pack is causing a truly ludicrous number of ColorHolders to be constructed. I'll poke at it and see if I can figure out what. It may just be that the clustering algorithm is slow on large images; that algorithm is meant to only ever be a fallback if my other system doesn't work, but it's getting triggered for every texture for that setup. Hmm.

Alright, and it looks like the expensive step that's slowing it all down might be calculating relative color distances? The clustering algorithm, on closer inspection, is very fast. The default algorithm, though, is calling ColorHolder#distanceToLab a lot, which is weird, so it looks like the speed issue is actually in the original algorithm, and is probably something that only comes up either when there's very few shared colors or when the image is very large. I'll see what I can figure out. The algorithm really wasn't designed for 128x128 images; in fact, I rather doubt that the extraction will work at all with modded ores with that resource pack installed. Plus, you're generating a bunch of 128x128 ore texture images... which fills up memory, likely leading to the crash. I'll look for any obvious culprits, but there's only so much I can do to fix this; the system simply isn't made for extremely high resolution resource packs, and in fact can't really be adapted to them super well without sacrificing the quality of extraction for smaller images. I'll see what I can do.

Yep, the expensive step is running the postCalcQueue, which in this case looks to be running for basically every pixel... which isn't too great, all things considered. That queue is basically "we didn't know what to do with this pixel the first time around, so let's try again" - it checks that pixel, and colors calculated from it, against every palette color in both images. Best I can tell it's running on many pixels in each image, and each image has a huge number of distinct colors, because of that resource pack. Not really too much I can do without sacrificing quality for normal-resolution texture packs I don't think.

Thank you for continuing to look into this. I've been keeping up with your responses but haven't had time to try anything as my son just wants me to play MC with him and not test mods lol.

So if I understand correctly, is it specifically that earthy texture pack? Or any texture pack 128 or greater?

Sorry if this sounds naive, but could the mod check the texture resolution first and then handle it differently depending on size?

Possibly; I suspect that the issue is not just the resolution but the number of different colors used. There's no good way to tell where to draw a boundary; I may end up doing some naive check based on the total number of calculations that would have to happen and just send it to the fallback algorithm if there's more than a certain number. Do you have other 128x resource packs you use? It's worked fine for me with 64x resource packs before, but those had very vanilla-like color setups.

Fix should be included when I publish a new version; you're still generating a lot of high-resolution textures, so it's going to be slow, but it should be more like 20-minutes slow. I'd turn on asset caching so that this only happens once - you'll need to delete your DynAssetGen config since there's been a format change. Still not sure why it ended up crashing on you - that sounds to me like you ran out of memory or something. If that's the case, there's nothing I can do - it may just be generating too many textures, which is unavoidable.

Alright, here's the main issue: it's extracted something like 500 foreground colors for each image, along with 74 background colors or so, and still has 500 pixels queued to be post-calculated. Which then takes a very long time. I'm going to set a cutofff of 10 million calculations, and just cull anything that wants to do more than that. I might decrease that cutoff later if I get more similar reports; maybe I'll make it a config option or something?

Alright, the clustering algorithm is still slow, but it's a lot faster than the normal algorithm for large images, so there's that. Should have a workable fix in the next version; not perfect, but about the best that can reasonably be done. Textures may occasionally looks weird if you have very high resolution resource packs with certain sets of colors, but I'm not sure how often that will come up

Hopefully mitigated in the new version; will still be slow but should eventually load, assuming you don't run out of memory first. Nothing I can do about that part of the issue though

Alright, made some more changes in 1.19.3 that should make this much faster