Carpet

Carpet

2M Downloads

[Scarpet] Support for reading Image files as an array of pixels

replaceitem opened this issue ยท 6 comments

commented

See a relevant discord discussion here.
This could be usefull for blockart stuff or similar.
I will probably make a PR on this soon.
@gnembon what about image writing? could be used to export maps of a server or stuff like that??
i could do both, if you feel like its a good addition to scarpet

commented

In any case a relevant methods to decode colors would be very welcomed (color2rgba and rgba2color) - could be done as a function, but that would be slower and useful anyways.

lol hypocrisy #374

commented

btw how will you do that? will you store as 2d array of NumericValues?

commented

Not quite sure yet how it would be the best to do (in terms of memory and cpu usage)
Either being able to read individual pixels form a file (though that would mean the file would have to be opened and closed for each read, but that would save memory and not use a huge list)
Or returning the full list, saving some cpu but being huge in memory
I'd love some opinions and advice on how that should be implemented

commented

well, I doubt you would be to run into memory issues with the usecases for pixelarts etc, where the images would be low resolution anyways.
And better memory optimized access via own methods (potentially defining its own container type would be clunky to implement. For these reasons I would lean towards image:x:y kinda access model (or :y:z, I am not discriminating against your coordinate preferences).
But, (and that's a big but), defining a new n-dimensional array of float types (tensors) (I would stick with doubles for practical reason, despite the industry swaying towards float32, float16 or even less to save memory and compute), as an efficient way of representing tensors, is something that I would be glad to see, potentially enabling neural networks and automatic inference in scarpet without being very inefficient. All of this while handling images as a sideeffect.

Another question is pixel color representation - would that be NxNx3(4) where each color has its own value, or NxNx1 where each value uses already existing standard for passing color values in shapes of 0xRRGGBBAA (as an int, perfectly contained in any double value). In any case a relevant methods to decode colors would be very welcomed (color2rgba and rgba2color) - could be done as a function, but that would be slower and useful anyways.

Obviously saving would work as well. In that case read_file and write_file would operate / require tensor types. Suggesting using more universal name tensor vs more specific to one implementation ndarray or something similar

commented

Actually if this gets implemented with support for tensors (started liking that even more), that would cover #593 partially, which is a very old suggestion.

commented

ooh, so you could have a dynamic way of representing the layers of a neural network! this looks cool!