How does "it works" go together with "it produces a lot of garbage"?
I understand that it also produces some correct (or correct looking) results, but the code sounds like it has lots of false positives from what you're saying, so it's probably not the correct algorithm.
As for the time it takes, it doesn't sound like something that should take so much time, but not sure how it's in the code.
I'd look for these things:
Is the DB storing the color info as CMYK? If so, maybe go through your existing table and add another table or extra columns with pre-computed columns for lightness, hue and chroma.
Then you can get a CMYK value as input, translate it to lightness, hue and chroma, and select only the rows where the query attribute (e.g. lightness) is different to the one you got. You can even apply the scale factor in the SQL query directly (e.g.
WHERE
CASE
WHEN $direction = 1 THEN lightness > $lightness + ($scale_ratio * $lightness)
ELSE lightness < $lightness - ($scale_ratio * $lightness)
END
Is the DB the slow part? Do you even need the DB? Perhaps you can loop through the colors (creating each in memory), and if you need to lookup some unrelated attribute like an assigned per-row id, you could query the DB for that after you found the N "matching" colors, and only for those.
Can you express the distance algorithm in SQL (or a PL/SQL function)? Is it faster?
A row in the DB has 7 columns: c,m,y,k,lightness,chroma,hue. I used the colorspacious python library to create the DB which took about 24 hours. I wanted to map all the colors in the CMYK gamut (about 10^8) over to the CIECAM02 model but I changed some of the math to make it populate the database faster and that DID introduce some error but the errors were on the order of 10^-6 and I don't know enough about CIECAM02 to tell if that's significant or not. Also I don't really have the time or the compute resources to test 10^8 colors. But I would make the DB again if I had some assurances that the math I was using was accurate and also I need to figure out how to parrallelize python because that would speed things up significantly.
I think a row in the DB is 7 floats, it should be 4 ints (c,m,y,k) and 3 floats (lightness, chroma, hue) but I don't know how to fix 10^8 rows in a reasonable amount of time.
The reason it takes so long is because first it searches for the source CMYK color. Then because of how floats work I can't just tell it to pick all the colors with the same hue, I have to specify some delta and pick all the colors where the absolute value of the hue minus the hue of the source color is less than the delta. That's the part that takes a lot of time but I can't really think of a way to speed it up beside changing the data types of the columns.
The reason it takes so long is because first it searches for the source CMYK color
Why does it need to find the source CMYK color in the DB? All the information from that row you either have or you can calculate without hitting the db, no? I mean, since your input is a CMYK color, you can just run the same mapping function on it and get the lightness/chroma/hue values corresponding to it - without touching the database.
Actually, do you even need the db for anything? If I understand correctly, it can go like this:
You are given an input CMYK values, a hue OR lightness OR chroma value and a scale factor,. Say you're given C,M,Y,K, attribute = "lightness" and scale_factor = 0.2 and direction = 1.
(a) convert the CMYK value you were given to H/L/C (running the mapping function).
(b) given a desired "smallest granularity" (e.g. one matching the precision you already have on the db), just run a loop on the H/L/C value you got in (a), incrementing L from its original value up to the max value you can go given the scale factor. For each step you have a new L value, say L' value, and you can combine it with H and C, to get: H, L', C. Convert that back to CMYK using a reverse mapping, and that's one of your answers. Increase L' another step and repeat.
Oh hey that's a good idea! Probably calculating the lightness, hue and chroma for the source CMYK color is going to be faster than searching for it in the DB. Also I'm using the DB because it provides mappings for all the CMYK colors. If I just work with CIECAM02, and run the math in reverse to get a CMYK color, there's no guarantee that the numbers that come out will actually be valid CMYK values: they might be greater than 100 or less than 0 and either case is not a valid CMYK value. This happens because CIECAM02 tries to model all the colors that exist in the visible spectrum and CMYK only represents all the colors which can be made by mixing cyan, magenta, yellow and black inks and thus there are fewer colors in CMYK than CIECAM02. Still I might be able to try "looking around" in the case that the computer calculates invalid CMYK and that might be faster than using the DB.
Yeah, you could maybe still create all the CIECAM02 values related to your input (within some granularity limit) and discard those producing values outside the CMYK gamut.
Or you can try "gamut mapping", on them: this is through algorithms to get the closest "narrow gamut color" in CMYK from a color in a "wider gamut color space" like CIECAM02.
Search for "gamut mapping algorithms" or "gamut compression".
Thanks! It turns out "gamut mapping" was the special keyword I needed to be searching for. Immediately I was able to find the coloraide library which does everything I need it to: convert colors from CMYK to CAM16-UCS, adjust the CAM16-UCS value by some perceptual attribute, map the new CAM16-UCS value back to CMYK and adjust it again if the new value does not fit in the CMYK gamut.
Comments
How does "it works" go together with "it produces a lot of garbage"?
I understand that it also produces some correct (or correct looking) results, but the code sounds like it has lots of false positives from what you're saying, so it's probably not the correct algorithm.
As for the time it takes, it doesn't sound like something that should take so much time, but not sure how it's in the code.
I'd look for these things:
Is the DB storing the color info as CMYK? If so, maybe go through your existing table and add another table or extra columns with pre-computed columns for lightness, hue and chroma.
Then you can get a CMYK value as input, translate it to lightness, hue and chroma, and select only the rows where the query attribute (e.g. lightness) is different to the one you got. You can even apply the scale factor in the SQL query directly (e.g.
Is the DB the slow part? Do you even need the DB? Perhaps you can loop through the colors (creating each in memory), and if you need to lookup some unrelated attribute like an assigned per-row id, you could query the DB for that after you found the N "matching" colors, and only for those.Can you express the distance algorithm in SQL (or a PL/SQL function)? Is it faster?
A row in the DB has 7 columns: c,m,y,k,lightness,chroma,hue. I used the colorspacious python library to create the DB which took about 24 hours. I wanted to map all the colors in the CMYK gamut (about 10^8) over to the CIECAM02 model but I changed some of the math to make it populate the database faster and that DID introduce some error but the errors were on the order of 10^-6 and I don't know enough about CIECAM02 to tell if that's significant or not. Also I don't really have the time or the compute resources to test 10^8 colors. But I would make the DB again if I had some assurances that the math I was using was accurate and also I need to figure out how to parrallelize python because that would speed things up significantly.
I think a row in the DB is 7 floats, it should be 4 ints (c,m,y,k) and 3 floats (lightness, chroma, hue) but I don't know how to fix 10^8 rows in a reasonable amount of time.
The reason it takes so long is because first it searches for the source CMYK color. Then because of how floats work I can't just tell it to pick all the colors with the same hue, I have to specify some delta and pick all the colors where the absolute value of the hue minus the hue of the source color is less than the delta. That's the part that takes a lot of time but I can't really think of a way to speed it up beside changing the data types of the columns.
Why does it need to find the source CMYK color in the DB? All the information from that row you either have or you can calculate without hitting the db, no? I mean, since your input is a CMYK color, you can just run the same mapping function on it and get the lightness/chroma/hue values corresponding to it - without touching the database.
Actually, do you even need the db for anything? If I understand correctly, it can go like this:
You are given an input CMYK values, a hue OR lightness OR chroma value and a scale factor,. Say you're given C,M,Y,K, attribute = "lightness" and scale_factor = 0.2 and direction = 1.
(a) convert the CMYK value you were given to H/L/C (running the mapping function).
(b) given a desired "smallest granularity" (e.g. one matching the precision you already have on the db), just run a loop on the H/L/C value you got in (a), incrementing L from its original value up to the max value you can go given the scale factor. For each step you have a new L value, say L' value, and you can combine it with H and C, to get: H, L', C. Convert that back to CMYK using a reverse mapping, and that's one of your answers. Increase L' another step and repeat.
Oh hey that's a good idea! Probably calculating the lightness, hue and chroma for the source CMYK color is going to be faster than searching for it in the DB. Also I'm using the DB because it provides mappings for all the CMYK colors. If I just work with CIECAM02, and run the math in reverse to get a CMYK color, there's no guarantee that the numbers that come out will actually be valid CMYK values: they might be greater than 100 or less than 0 and either case is not a valid CMYK value. This happens because CIECAM02 tries to model all the colors that exist in the visible spectrum and CMYK only represents all the colors which can be made by mixing cyan, magenta, yellow and black inks and thus there are fewer colors in CMYK than CIECAM02. Still I might be able to try "looking around" in the case that the computer calculates invalid CMYK and that might be faster than using the DB.
Yeah, you could maybe still create all the CIECAM02 values related to your input (within some granularity limit) and discard those producing values outside the CMYK gamut.
Or you can try "gamut mapping", on them: this is through algorithms to get the closest "narrow gamut color" in CMYK from a color in a "wider gamut color space" like CIECAM02.
Search for "gamut mapping algorithms" or "gamut compression".
Thanks! It turns out "gamut mapping" was the special keyword I needed to be searching for. Immediately I was able to find the coloraide library which does everything I need it to: convert colors from CMYK to CAM16-UCS, adjust the CAM16-UCS value by some perceptual attribute, map the new CAM16-UCS value back to CMYK and adjust it again if the new value does not fit in the CMYK gamut.
https://facelessuser.github.io/coloraide