As someone who draws, there are some obvious aspects where AI generation just works perfectly:
- generate details and texture, which photobashing was already used for; but that mostly solve the licensing problem for it
- generate random inspiration boards, for which image search was used (again, mostly solve the licensing problem)
- generate derivative stuff, e.g. typical game portraits or props
In practice, only the third point is generally discussed, because it lowers tremendously the entry barrier to generate images for people without any skills. It's like if you could pick up screenshots from other content, clean them up, and you're free to use it.
Whereas it essentially does not work for:
- cartoony generation. It relies too much on line consistency, visual clarity and abstraction
- concept design --not the flashy 10 minutes speedpaint type, but where you have to combine ideas in meaningful ways. In particular hard-surface design which required good 3D thinking and consistency of the whole.
These fundamental flaws are omnipresent, but can be hidden by certain styles where things are implied by color blobs, hidden by stylized brushstrokes, or simply an overflow of details (something Midjourney is very good at).
All in all, it feels like AI is a danger for people at the bottom of the profession hierarchy, but will elevate people at the top, whose work cannot be replaced. In other words, people who are more akin to be considered "artisans" rather than artists, who will take a prompt and simply clean it.
In particular, drawing has something like the 20/80 rule, where all the creative input is in the first 20% and the rest is 'rendering', a very mechanical task which you can mostly do with your brain turned off. As Yumenoley put it, "it was a mistake to let the AI do the interesting part".
Cartoony generation is probably a matter of training on the appropriate source material, for what it’s worth.
https://dreambooth.github.io shows a glimpse of the future. You’ll be able to upload a few drawings that you want to emulate (e.g. Mickey Mouse), and then you can give it a specific prompt (e.g. Mickey Mouse doing a handstand).
You’re probably right about the consistency of 3D concepts, though. On the other hand, I was going to say “If you need a specific table, AI might not be able to help” — but again, dreambooth shows that we might be able to upload a few photos of a certain table, and it’ll take care of the details.
Give it a few years. :)
I think you’re spot on that AI will be an incredible tool for artisans. I used it to make some video game music: https://soundcloud.com/theshawwn/sets/ai-generated-videogame... Even though I can’t play any instruments too well, I was able to craft each piece uniquely. (My favorite is “Crossing the Channel”, which has a strange rhythm because I’m pretty sure the AI made a mistake at the beginning, and then extrapolated the next “actually, this isn’t a mistake” song that it thought of, which turned out to sound cool. A bit like a guitarist doing improv.)
I think the main issue with cartoony generation is that professional animators, are not being tasked with "here is famous character Mickey Mouse that's omnipresent in your source data, now draw him on the moon wearing a hat", they're being tasked with something where the source material is a couple of concept images possibly at lower fidelity than the desired end output (and the task might well be much more specific, and the art directors considerably more pedantic about the quality and consistency of the lines than the average hacker typing in magic incarnations to get something resembling fantasy art for their blog). Of course, if you want to make memes of Mickey Mouse wearing a hat on the moon, or Mickey Mouse at the White House with a hammer and sickle, or an anime-style Mickey Mouse, AI cartoons may well be plenty good enough already.
There's a role for AI in filling in gaps, but illustrators remain essential in creating the basic style to extrapolate from and even more so in professional quality work (And to some extent, other procedural generation techniques were able to fill gaps up before cutting-edge NNs - the stampede of wildebeest in the 1994 Lion King was procedurally generated from a handful of models for example)
Agree that dreambooth is the future. I'm building an app that lets users with no technical background train their own dreambooth models for $2-$4: https://synapticpaint.com/dreambooth/info/ They can also share their trained models for others to use.
Here are some potential use cases:
- for fun (giving yourself a makeover, inserting yourself into famous movies)
- cheaper way to get studio photos (wedding photos, professional headshots for actors/models)
- easy way to create marketing assets. Like if you own an etsy store and don't want to engage a marketing studio, instead just create a dreambooth model of your necklace or whatever and create high quality product photos
There are probably a bunch of other use cases. I think making this easy (no figuring out how to do a git pull or rent a gpu) plus the community sharing aspects will make this technology a lot more accessible to artists and general users, and then the users will be doing all kinds of cool things with it organically.
All in all, it feels like AI is a danger for people at the bottom of the profession hierarchy, but will elevate people at the top, whose work cannot be replaced.
The question is whether it not rather puts the people in the middle under pressure, when it helps the people at the bottom to produce better quality.
I can observe such a shift in translations. Deepl is not perfect, but it allows me to improve my own English texts considerably. Even if I were to give my text to a professional to polish, she or he would have much less to do than before, when I was not assisted by Deepl.
One thing I think needs to be looked at a little bit more deeply is that the AI is not stopping where it is today, it's going to get better and if it targets the lowest part of the market now, what's it going to do in 10 years?
This reminds of of the chess computers from 30 years ago. Experts were convinced computers would never beat GM humans, based on the state-of-the-art of the time. They never took into account all the future advancements in hardware and software.
The chess experts were wrong qualitatively. They were saying that best play was about intuition and feel, that computational power would never replicate regardless of its scale. (Data on Star Trek always lost chess matches implausibly.) Turned out that brute force calculation thirty moves deep does in fact outperform anything a human can foresee.
The same thing happened for both Go and Starcraft in the next decades, the experts said computers couldn't replicate enough spatial feel, and then they did. And now it's happening for AI art. Enough computational power and a sufficiently well-trained neural network can indeed exceed anything a human can do.
AI has been roughly doubling in performance every year or two, for quite some time. We just never noticed when it went from 0.0001% to 0.0002% of human capability. This is the year that it doubles from 10% to 20% and everybody notices. And there's not a lot of doublings left until it shoots past 100%.
All in all, it feels like AI is a danger for people at the bottom of the profession hierarchy, but will elevate people at the top, whose work cannot be replaced. In other words, people who are more akin to be considered "artisans" rather than artists, who will take a prompt and simply clean it.
I am an artist who is in a place where she gets paid decently to draw whatever the heck she feels like, with little regard for commercial viability.
The jobs you're dismissing as mere "artisans" are the ones where I was able to be in a place where I could spend most of my waking hours honing my drawing skills and still pay my rent. Every time you draw a thing, you get a little better at drawing that thing, and a little better at drawing in general. This is how you master your craft. If you're part of a studio then even better: there's older artists above you, and tons of opportunities for them to critique your work and open your eyes to the major flaws in it you can't see yet. If AI art fills all those niches for expanding on someone else's prompts, budding pros will find it a lot harder to get to the point where this virtuous circle of being paid to practice their craft gets started.
In particular, drawing has something like the 20/80 rule, where all the creative input is in the first 20% and the rest is 'rendering', a very mechanical task which you can mostly do with your brain turned off.
This really depends on your process. A big part of becoming a pro, in my experience, was finding ways to make the boring parts happen a lot faster, and giving myself more opportunities to do the fun parts at every stage of the piece. That said there's also a pleasure to be found in putting on some good music, turning your brain off, and rendering the heck out of something! It's that much-coveted "flow state" people love to talk about.
I think all in all AI is a danger for everyone in the industry and there's no reason to sugar coat it.
Pure economics makes whatever hopeful expectation of advantage created by the AI generated medium irrelevant for anyone with a livelihood in the field. Being the creative genius may allow you to retain your employment, but rest assured, your income growth will decline and the next generation coming after you will be making less. The value of graphics will decline and so the money earned by the people making it will also decline. You're making truly creative work? Great, a company can just take your rendering and use it as the basis for an AI generated image to remove any licensing requirement. Good luck trying to prove original ownership. And who needs creativity anyway? How often are graphics used in commercial/retail/media enterprises really breaking the mold? Whatever shortcomings mentioned are already being mitigated by add-on AI systems (Tencent's ARC for instance) so there should be an expectation of rapid improvement in the coming years.
People employed in the field will likely adapt away from non-lucrative image generating work eventually, but the transition will be pretty painful and the overall effect will likely constrain incomes for creative work in general.
Comments
As someone who draws, there are some obvious aspects where AI generation just works perfectly:
- generate details and texture, which photobashing was already used for; but that mostly solve the licensing problem for it
- generate random inspiration boards, for which image search was used (again, mostly solve the licensing problem)
- generate derivative stuff, e.g. typical game portraits or props
In practice, only the third point is generally discussed, because it lowers tremendously the entry barrier to generate images for people without any skills. It's like if you could pick up screenshots from other content, clean them up, and you're free to use it.
Whereas it essentially does not work for:
- cartoony generation. It relies too much on line consistency, visual clarity and abstraction
- concept design --not the flashy 10 minutes speedpaint type, but where you have to combine ideas in meaningful ways. In particular hard-surface design which required good 3D thinking and consistency of the whole.
These fundamental flaws are omnipresent, but can be hidden by certain styles where things are implied by color blobs, hidden by stylized brushstrokes, or simply an overflow of details (something Midjourney is very good at).
All in all, it feels like AI is a danger for people at the bottom of the profession hierarchy, but will elevate people at the top, whose work cannot be replaced. In other words, people who are more akin to be considered "artisans" rather than artists, who will take a prompt and simply clean it.
In particular, drawing has something like the 20/80 rule, where all the creative input is in the first 20% and the rest is 'rendering', a very mechanical task which you can mostly do with your brain turned off. As Yumenoley put it, "it was a mistake to let the AI do the interesting part".
Cartoony generation is probably a matter of training on the appropriate source material, for what it’s worth.
https://dreambooth.github.io shows a glimpse of the future. You’ll be able to upload a few drawings that you want to emulate (e.g. Mickey Mouse), and then you can give it a specific prompt (e.g. Mickey Mouse doing a handstand).
You’re probably right about the consistency of 3D concepts, though. On the other hand, I was going to say “If you need a specific table, AI might not be able to help” — but again, dreambooth shows that we might be able to upload a few photos of a certain table, and it’ll take care of the details.
Give it a few years. :)
I think you’re spot on that AI will be an incredible tool for artisans. I used it to make some video game music: https://soundcloud.com/theshawwn/sets/ai-generated-videogame... Even though I can’t play any instruments too well, I was able to craft each piece uniquely. (My favorite is “Crossing the Channel”, which has a strange rhythm because I’m pretty sure the AI made a mistake at the beginning, and then extrapolated the next “actually, this isn’t a mistake” song that it thought of, which turned out to sound cool. A bit like a guitarist doing improv.)
I think the main issue with cartoony generation is that professional animators, are not being tasked with "here is famous character Mickey Mouse that's omnipresent in your source data, now draw him on the moon wearing a hat", they're being tasked with something where the source material is a couple of concept images possibly at lower fidelity than the desired end output (and the task might well be much more specific, and the art directors considerably more pedantic about the quality and consistency of the lines than the average hacker typing in magic incarnations to get something resembling fantasy art for their blog). Of course, if you want to make memes of Mickey Mouse wearing a hat on the moon, or Mickey Mouse at the White House with a hammer and sickle, or an anime-style Mickey Mouse, AI cartoons may well be plenty good enough already.
There's a role for AI in filling in gaps, but illustrators remain essential in creating the basic style to extrapolate from and even more so in professional quality work (And to some extent, other procedural generation techniques were able to fill gaps up before cutting-edge NNs - the stampede of wildebeest in the 1994 Lion King was procedurally generated from a handful of models for example)
DreamBooth works with just 4 input images.
Agree that dreambooth is the future. I'm building an app that lets users with no technical background train their own dreambooth models for $2-$4: https://synapticpaint.com/dreambooth/info/ They can also share their trained models for others to use.
Here are some potential use cases:
- for fun (giving yourself a makeover, inserting yourself into famous movies)
- cheaper way to get studio photos (wedding photos, professional headshots for actors/models)
- easy way to create marketing assets. Like if you own an etsy store and don't want to engage a marketing studio, instead just create a dreambooth model of your necklace or whatever and create high quality product photos
There are probably a bunch of other use cases. I think making this easy (no figuring out how to do a git pull or rent a gpu) plus the community sharing aspects will make this technology a lot more accessible to artists and general users, and then the users will be doing all kinds of cool things with it organically.
The question is whether it not rather puts the people in the middle under pressure, when it helps the people at the bottom to produce better quality.
I can observe such a shift in translations. Deepl is not perfect, but it allows me to improve my own English texts considerably. Even if I were to give my text to a professional to polish, she or he would have much less to do than before, when I was not assisted by Deepl.
One thing I think needs to be looked at a little bit more deeply is that the AI is not stopping where it is today, it's going to get better and if it targets the lowest part of the market now, what's it going to do in 10 years?
Agreed. AI art will continue to improve.
This reminds of of the chess computers from 30 years ago. Experts were convinced computers would never beat GM humans, based on the state-of-the-art of the time. They never took into account all the future advancements in hardware and software.
The chess experts were wrong qualitatively. They were saying that best play was about intuition and feel, that computational power would never replicate regardless of its scale. (Data on Star Trek always lost chess matches implausibly.) Turned out that brute force calculation thirty moves deep does in fact outperform anything a human can foresee.
The same thing happened for both Go and Starcraft in the next decades, the experts said computers couldn't replicate enough spatial feel, and then they did. And now it's happening for AI art. Enough computational power and a sufficiently well-trained neural network can indeed exceed anything a human can do.
AI has been roughly doubling in performance every year or two, for quite some time. We just never noticed when it went from 0.0001% to 0.0002% of human capability. This is the year that it doubles from 10% to 20% and everybody notices. And there's not a lot of doublings left until it shoots past 100%.
I eagerly await a fully ML generated ballet choreography with a dozen participants. Maybe 15 years from now.
The super hard problem is driving a robotic body, vs rendering an animation of the above.
I am an artist who is in a place where she gets paid decently to draw whatever the heck she feels like, with little regard for commercial viability.
The jobs you're dismissing as mere "artisans" are the ones where I was able to be in a place where I could spend most of my waking hours honing my drawing skills and still pay my rent. Every time you draw a thing, you get a little better at drawing that thing, and a little better at drawing in general. This is how you master your craft. If you're part of a studio then even better: there's older artists above you, and tons of opportunities for them to critique your work and open your eyes to the major flaws in it you can't see yet. If AI art fills all those niches for expanding on someone else's prompts, budding pros will find it a lot harder to get to the point where this virtuous circle of being paid to practice their craft gets started.
This really depends on your process. A big part of becoming a pro, in my experience, was finding ways to make the boring parts happen a lot faster, and giving myself more opportunities to do the fun parts at every stage of the piece. That said there's also a pleasure to be found in putting on some good music, turning your brain off, and rendering the heck out of something! It's that much-coveted "flow state" people love to talk about.
I think all in all AI is a danger for everyone in the industry and there's no reason to sugar coat it.
Pure economics makes whatever hopeful expectation of advantage created by the AI generated medium irrelevant for anyone with a livelihood in the field. Being the creative genius may allow you to retain your employment, but rest assured, your income growth will decline and the next generation coming after you will be making less. The value of graphics will decline and so the money earned by the people making it will also decline. You're making truly creative work? Great, a company can just take your rendering and use it as the basis for an AI generated image to remove any licensing requirement. Good luck trying to prove original ownership. And who needs creativity anyway? How often are graphics used in commercial/retail/media enterprises really breaking the mold? Whatever shortcomings mentioned are already being mitigated by add-on AI systems (Tencent's ARC for instance) so there should be an expectation of rapid improvement in the coming years.
People employed in the field will likely adapt away from non-lucrative image generating work eventually, but the transition will be pretty painful and the overall effect will likely constrain incomes for creative work in general.