Is this AI Art, or is This Something New?
Insertar
- Publicado el 25 sep 2023
- Watch Anime Rock Paper Scissors 2 LIVE NOW on CorridorDigital.com/
Our videos are made possible by Members of CorridorDigital, our Exclusive Streaming Service! Try a membership yourself with a 14-Day Free Trail ► corridordigital.com/
MORE Exclusive Tutorials Coming Saturday, August 12th! ► until then watch how you can make your own anime (or anything really) with our click-by-click tutorial: www.corridordigital.com/video...
Take a look under the hood to see how the use of new machine learning programs is shaping the way we created Anime Rock Paper Scissors 2, and the new processes we're inventing to harness the power of these incredible tools.
Creative Tools ►
Puget Computers: bit.ly/Puget_Systems
Aputure Lights: bit.ly/Corridor_Lights
B&H Photo: bhpho.to/3r0wEnt
ActionVFX: bit.ly/TheBest_ActionVFX
Cinema4D: bit.ly/Try_Cinema4D
Nuke: bit.ly/Nuke_Compositing
Houdini: bit.ly/HoudiniSims
Octane Render: bit.ly/Octane_Wrender
Epidemic Music: bit.ly/Corridor_Music
Chapters ►
00:00 Birth of a New Art Form?
01:12 Problems with Unrefined Tools
02:57 Building Solutions
04:25 Addressing the Controversy
05:32 How We Make a Scene
08:59 The Finished Product
10:05 A New Art Form Entretenimiento
The training on particular characters from your own art has made for a dramatic improvement! However, it's noticeable how primitive the mouth animation is in this iteration - there's very little shaping of the lips, only opening and shutting of the mouth. I wonder if you had got Josh to draw a lot more examples of character mouths saying various phonemes whether they would have come out better.
In their first video, they said they did basic puppet Mouth opening as an anime style choice.
Yeah the mouths look really bad. Can't beat traditional animation.
@Jordan Holmes that's just the style they're going for
@Jonah Nelsonyeah but the audio syncing with the lip flaps could be better. I know, baby steps but this goes to show that even human animators are still needed for details AI does not grasp in finer details.
@Reagan Peterson but as they mention in to the first one it has more of a rotoscoped feel, so it ended up with more lip shapes getting through
Exciting to see the progress!
These behind the scenes videos are just as exciting to me as the show. The passion behind the project is palpable.
I really appreciate Corridor for addressing the rising controversy around AI with their own exploration from the position of artists themselves and finding a way to use it effectively, ethically, and while preserving their own expression and style and skill in their pieces.
people on twitter are probably still going to find reasons as to why this isn’t art and it’s actually stealing 🙄🙄🙄
Addressing, yeah, I'd say so. But clearly, they say their stance on the whole thing is, "If you like it and an AI made it, then we think it's fine."
I loved the jittery art style because it literally looked like just that, a style.
It reminded me of 'a scanner darkly'.
And since the last ep I've been curious what a scanner darkly could have looked like had it used this Ai technique instead of the hand drawn stuff.
Could be an experiment for a future project maybe? Scanner darkly with todays technology
Much of my issue with AI art is the uncredited/unpaid use of source artists work (after which many suggest it is their work). In this case, an artist was paid to generate the training data for your model. That feels like the morally correct way to use AI.
That's not quite true. What they undoubtedly did is use the extremely general base AI that was trained on billions of images (most of which are photos, not drawings btw), and then finetune it with the style provided by this artist to make it consistent. The use of the artist wasn't because it makes it ethical, it's because the artist is needed to bring consistency. While it's possible to train a new AI on art deliberately supplied for training, we don't have that yet. We'd need at least tens of thousands of consenting artists, photographers, people just running around streets with cameras, to get an AI model I think most people would consider ethical. Overall, I thing they're taking steps in the right direction and making the most of what they have. I hope they have access to more ethical tools in the coming years.
@Empress Lithia Ah, yeah that would make sense. A step in the right direction .. indeed I agree.
You could circumvent that by training an AI model on textual descriptions of what an artist's style looks like, only pretraining the model using images that you have the rights to use.
I take issue with the judgement that training the AI is not ethically done unless you train it on your original work. As long as it adhered to the concept and content legally seen as FAIR USE it does not matter what and whose content was used and how much. Copyrighted material is often used for inspiration, for teaching and learning on your own. If somebody writes a book about a subject I like in a style I like and then write my own book about the same thing in the same style as long I don't use the same exact words in the same sequence It would perfectly within my legal right to do so. As long as people are honest about their influence and are not trying to closely copy something it's all good. Subtle or big differences will naturally appear due to different skills, tastes, needs and environment. Also if I read 3 books that inspire me to write one book containing different aspects of those books. nobody expects me pay royalties to the other authors unless I use exact elements in the exact way, why are people arguing that AI should be different, not to mention that even if we did this with AI most authors used for training would deserve to get a fraction of a fraction of a dollar at most.
The way copyright is now used is rather perverse and unethical. It often just protects people and companies that have the money and time for lawyers and litigation and does little to protect common creators from non authorized use. Long copyright "protection" even makes very hard saving some older works for which their copyright holders can't be found.
I’d love to see this built on with additional training for lip syncing. Like adding a library of animated expressions and mouth shapes (similar to the old school stuff they made for the back end production on The Nightmare Before Christmas) and see what kind of boost we can get from that. Plus, the challenge of figuring out how to implement that in the best ways would be interesting.
There's already software to adjust mouth movement for dubbed audio or fixed lines. Not sure how well it works on animation as I've only seen it for live action, and no idea the licensing cost, but the software solution is already out there.
It was a stylistic choice. They purposefully just opened and closed mouths, not articulating the words.
I'm basically repeating what other people have said, but... They recorded the audio separate from recording the video. In the video recording, the actors did not try to speak the lines, they just made extreme mouth movements synced to the lines. The open mouths, snarls, etc. are exaggerated in anime so when they did their video pass, they tried to do the exaggerated mouth movements rather than actually talking. If the mouth movements don't sync with the audio, it's because of the ACTORS not syncing - it's not a software problem.
@Boring_Ugly_Dude Yeah, I’m fully aware of all of that. I wasn’t referring to perfect lip syncing like some Disney animation, but to achieve a somewhat better repertoire of shapes beyond ventriloquist dummy status. All it needs is a couple variations of mouth shapes, even if it doesn’t perfectly sync to every word, but simply some variation of mouth shapes to shuffle in there at least.
@靄 MOYA HORROR There were a lot of shots in this episode where the characters mouth barely move if at all when they're speaking
It looks great, one of the things I notice is the eyes still distort and break. We need a way to generate the eyes an have it stick to the same shape. One of the idea I had was to make clean versions of all the eyes from each angle, and have a seperate AI pass that just does eyes. Hopefully it can track them better with most angles and pupil directions. The other thought was to have the eyes rendered with a 3D AI, that actually makes the physical eyes and lighting based off the drawings, then applies them to the characters appropriately. Gen2 is currently doing some nutty stuff with video shots. Especially with stable models from frame to frame.
Proud of you guy's for inventing your own art style. (If there is anything left to invint really.) But still, you didn't have to but you cared enough to listen to the people. Despite how crazy some of us can get. 😉
I really like the artstyle Josh came up with. very contrast, vibrant, and clear.
Still not half as good as Bloodlust tho... But I prefer it this way
@Oliver Falcó the guy said he'd never made anime before, so considering that I think he did an excellent job.
I personally LOVE the new artstyle, it's such a great blend between anime and cartoon, which seems perfect when using tools that struggle with clarity and consistency :D
I would be fascinated to see a breakdown of how long it takes to do this, versus a studio doing a more traditional animation technique. Obviously the Crew is still feeling out the work flow and everything, but the time comparison would be interesting i think.
I think it would be largely based around two large parts: 1) time to train stable diffusion with source material, and 2) time to export video shots into individual frames, run them through stable diffusion and recompile them
@Spiker985 Studiostraining speed can be solved by giving it more power. The slowest part is creating a dataset, describing every single image and everything contained in it. And then doing it again when training goes wrong.
I trained some loras on datasets of two dozen images and even that was pretty tedious and slow process
But., everytime there are a new tools to make the workflow faster,
all the big studios will use it as a license to make a half assed product,
new workflow mean that they could McDonalds-ify the product,
make a mid story, mid vfx, and assign a cheap, clueless and mid people to create the product
.
fast production mean fast money, therefore it is not about the Product anymore,
it's about making a generic product that addressing the current trends
because they saw that technology as a way to replace expensive people in their company
we already saw this phenomenon time and time again, in the movie industry, in the game industry.
there are a massive lost in the knowledge base, because the guy who create the masterpiece was too expensive to hire,
and the knowledge wasn't getting passed down to the next generation.
therefore we saw a massive degradation of quality in the past 10 years
despite the promise of giving you the power to make a jawdropping scene and great storytelling easier,
giving people the liberty of creation.
we saw less and less Masterpiece each and every year
@Jensen Raylight the big studios were doing exactly this for years, well, effects were good but story usually worse then mediocre to compensate for this:)
From what I gathered the AI in the 1st animation grabbed info from VHD and that style often uses squint eyes, closed eyes then animating them wide open in times of shock surprise etc, this is a key style while the AI couldn't allocate where these changes should accour, so even tho you could take care of the flicker fixing the face is a nightmare.
So if you are going to train something I guess a code needs to exist that covers opening and closing the eyes, because it pulls it all over the place at random and if you spend time fixing it frame by frame might as well make everything from scratch since you will get more clean results lightyears better in comparison and original style too.
The time spent training, editing rendering versus the time making everything from scratch by an capable artist burns less money and less resources.
I honestly don't want to know what these people pay their electricity bills on all these badly optimized amateur programs and AI scripts ...
A year ago I was playing around with learning AI things and not only it nearly fry my SSD, RAM, GPU and CPU basically whole PC was burning, my electricity bill was 100 times more then during winter time heating ... I had to stop burning time and money also had to repair my PC's cooling asap.
With other words I burned years worth of winter heating in just few weeks time.
So yeah no thanks ... I rather do nothing or crowdfund and pay actual artists if I want something done properly ... F--k AI and these amateur programs too ...
The AI obviously still struggles a lot with the mouth animation, but it's a lot easier on the eye without the flickering lines. Also I'm really glad that you used the drawings of an artist who fully knew about the project and consented to it, because that's what usually makes me mad about how people use AI art. I'm still sceptic about AI art, especially for commercial use, but I know we can't put the cat back into the bag, it's a thing now and it's impossible to stop - so I think what you are doing now is a move into the right direction. I really hope we'll soon find a way to make it fair for artists.
Edit: To clarify, I don't mean "slap copyright on styles so you can sue anyone who creates something close to it". I mean "let artists keep their jobs and make consent a defining factor in training AI." Would be nice if we could find a way to make that work.
Do you pay a license to the guy who designed and made the first snare drum in order to reproduce a specific sound? What this AI is mostly doing is learning how to recreate brush strokes, its not photoshopping someone elses work onto something else. For sure if the ai produces something that looks almost exactly like someones art you may have an argument but that basically doesn't happen because the dataset is too large.
@WARnTEA Don't forget about how the amen break is used in nearly every single genre in existence and yet the man who made it got nothing
people are really hypocritical to their own circumstances.
@WARnTEA For digital instruments / samples, yes, you do pay for them to have them reproduce that sound for you. (as far as I know)
Also, with a lot of modern AIs, you can tell them to generate things in the style of a particular artist and they will do it. (but the effectiveness varies based on the artist)
But my personal perspective is that most of the current AIs are a product of taking massive amounts of material without having the rights to use it (let alone in a commercial setting) and using them to create a for-profit product which, to me, sounds at least questionable.
@Sammysapphire Because you can't copyright a drum fill. Someone else came up with it before it was done by greg coleman.
@Cameron Yea but you are paying the person that hit the drum with a stick one time and recorded it. You aren't actually paying the person that designed the sound and built the drum. The only reason you are paying them is because they did work to assemble the sounds in one place, and they convinced you to buy it. No one is getting sued for recording their own sample of the exact same drumhit. It'd be like an artist drawing a bird and then suing anyone that draws a bird. These things exist in the real world, they are building blocks of life, they are not art on their own they are just things. It only becomes art when humans arrange those building blocks in a specific order and add a deeper meaning to it.
you know what? god bless yall and i love seeing people work on something they're passionate about. you're hardworking and you'll be the one to break through this new gen of production and animation
I feel you guys have been a lot more creatively free and playful with your compositions and scenes thanks to this new work method, It will be interesting to see where you go with it and if you could blend it with your "traditional" way of making videos.
Absolutely love this pipeline! Keeping the artists involved improves it so much 😍
The directing and effects still are art, thats stuff still takes some talent.
I'm neutral with AI and I love how you're handling this! Making your own art was definitely the right call but I would still love to see more articulated mouths because I think being able to do those would elevate this technology even more.
Regarding the mouth animation, they actually did that on purpose! They've said in a previous video (Did We Just Change Animation Forever?) that they purposefully didn't act out the lips to make it look more cartoony
Doesn't matter you are neutral or not
ai art still a threat to creative jobs
@Blopp How? Companies trying to remove large swaths of their workforce? IMO that sounds like ... a company / corporate problem to me.
So cool that you guys listened to the main criticism and really went the extra mile to make this project your own rather than pulling from and already existing style.
Whatever AI art become real things in future or not, you guys is making history as one of early AI arts adaptations.
Very impressive and intriguing. Just showed my wife both episodes and she loved it.
I appreciate you guys doing your own art this time major improvements
I will never get tired of watching this channel.
Yep
I never get tired of your mom
wait till you hear about their website
Facts.
AI has yet to discover what a tounge is. Artists remain safe for another day.
i think part of the controversy is that people tend to think that AI is a replacement, that the hard work of artists are now void because of how easy AI does it in comparison.
but the tools you used in RPS 1 and 2 are not for replacement, its for improvement, a tool that does things better.
if digital art is an improvement to traditional painting
then the AI art animation you did is an improvement to traditional animation
Thank you for using commissioned artist work that they gave their consent for to be used in this project. This is an ethical use of AI and should be the standard.
Except someone will just hire an army of third world artists at 50c/week to make art in the style of well-known artists and then we're back in the same place.
@Chaos Corner look it's a start, i understand that there is a whole system of using studios with underpaid workers, and it's definitely something that needs to be addressed. For now I will give these guys a bit of credit for doing something right. We'll then work on unionizing animators around the world.
DId their artists get the consent of the Vampire D animators to study their works?
@CaliMeatWagon I think they kinda covered that in their explainer video. I may have to double check it now again to see how they were able to work around that since they filmed themselves before rendering those shots into animated frames as well as using stock photos of castle interiors as opposed to using a prompt to magically yield out a frame.
@CaliMeatWagon You are aware that every artist ever looks at other pieces of art for inspiration right? No one is just sitting in a bubble thinking of every little thing from scratch. That's literally why trends exist.
Part II is by far visually more appealing in it's consistency. I Love both but this new version is perfect!!
love how you guys are always pushing the medium, inspiring!
I'm positive this will be the industry standard within studios to avoid copyright issues.
The improvement on the sequel is absolutely wild.
Keep pushing the boundaries crew 💪
A great informative video, as always guys. Looks great.
_Rock, Paper, Scissors_ always gave me huge _A Scanner Darkly_ vibes. The style may be different but as an animation tool it's basically very advanced rotoscoping, and I see how it can lead to new and exciting art works.
I also don't see it replacing traditional animation. I mean, can it be applied to the style of like _Futurama, Samurai Jack,_ or _Bojack Horseman?_ It woiuld be interesting to try just for the fun of it. But other tools and techniques will always be around. Photoshop didn't replace oil paints either after all.
What I dislike on the ethical side are the passive income bros who are touting AI as a way to "democratize art" while pumping out derivative kitsch. You're not democratizing anything, you're replacing real artists with machine learning. Not the same vibe at all.
Really appreciate that you guys created your own training data
100% believe this and several other of your videos are used in schools around the world. Amazing work.
Could you simply train a separate models for each element of the face and then combine them together? That way you could sharpen the eye models as an example by having an eye-specific training set or mouths, etc. Then use a separate machine learning model to combine the elements. It adds an extra layer of machine learning, but I would be curious to see if it generates a cleaner end result. I freely admit this is likely to be massive amounts of extra work, but it is an interesting thought experiment.
The original art style was far superior but I understand why you guys needed your own. Very cool to see. Keep it coming!
I do enjoy the previous art style more because I think it looks more gritty and visceral and I love that rotoscope look, but I appreciate the reasons why you changed it
The flickering on the first one sold it more for me. More of a human touch. New looks like a cartoon style on a 3d animation.
@Jacksalssome or an anime style on a cartoon
Agreed, it felt more "real" vs a cartoon trying to be an anime.
All previous "flickering" remembers me experimental European animation of years ago...
I think if they had a larger data set to train their model on, the video could have come out better. The old Anime Rock, Paper, Scissors had potentially thousands of frames to learn from. I can't imagine the artist they worked with drew more than a few dozen images.
I liked that you guys hired an artist to invent your own style. I think that over taking the work of an other artist is great. this video is great because it creates a good synthesis between ai and art.
THANK YOU for using dubbed style and paying homage to its eastern roots. its can be confusing those that haven't watched the style & genere, but is on brand for your creative style and team. Thank you for your art.
This makes me think of motion capture for 2d animation. Definitely a lot of potential when used correctly.
The creative juices are strong with this one 🤩
I appreciate that they’re not copying another art style through ai, but at the same time I do think the art style outputted from the ai looked much better, more detailed and serious, than their own new style, even if the flickering itself was improved
I agree with you, but Vampire Hunter D´s model is so much more complex. They could hire a veteran anime artist to create a dataset for them, but the cost would be much higher, and they´re taking the first baby steps in a new workflow. I assure you we´re going to see such datasets from the big companies like Bandai very soon.
To me this is the crux of why compensating the creators of the source data is important. With full respect to the artist who created their new style the work in vampire hunter D is stellar and the result of professionals at the top of their field. The ai output is only as good as the source data you give it so it does have value. If people want to train models based off the work of an artist that artist should be compensated for it, which then allows them to create more art which can then contribute to other models. There's a real possibility for everyone to benefit here.
I find it especially intriguing that with a solid pipeline, projects certainly seem so achievable that were never previously. It really let's your creativity go wild with the possibilities.
It could even be as draft or a strong pitch for a much deeper, more expensive, and involved full feature project.
You could try using a bilateral filter for smoothing the original video while preserving edges. Theoretically this makes stable diffusion even more stable because there are less details/noise
100% agree. I have never been artistically talented and never thought I would be. I had dreams and ideas but was never able to articulate them. AI tools have allowed me to bring expression to those ideas I thought would have to die within me.
Tools like these are always met with disdain at first, in the same way that the photograph was thought of as a false medium for people who weren't talented enough to become painters.
Thank you for speaking some common sense in pointing out that anything that gives people the ability to express themselves artistically can be used for good.
The fact that this is trained on custom artwork and not artist who haven't consented puts every fear to rest. This is an incredibly powerful tool that still takes amazing skills and artistry to use, and if emotionally impactful art can be created with it then that's excellent in my book.
Exactly my feelings. Train off your own character sheets
Wholeheartedly agree! Now, Could you imagine the results if Corridor Crew had used character sheets created in 3D? 😲
@DannyVdesignThis. Anything involving illegal scraping is still unregulated theft.
Art is free. There is no consent.
@Bedinor No.
Well it’s finally happened. Major CC content locked behind a paywall. I can only hope it eventually comes to ESclips.
the eyes still breaks the magic for me but it's quite cool to see how far did you come from rps 1, great job everybody
Yeah indeed. I would opt to redo them manually, at least for the pupils.
@pendaco Well they are still just playing around with the tech. The aim is to improve the process while making a cool interesting video They are not trying to make a perfect anime product that actually compeat in the normal way. And doing things like hand animating eyes would be a large sink of time, energy and money when that kind of perfection is not the goal. The selling point of the videos for now is the ai novelty not the art alone.
The eyes, the shadows, the teeth are still really wonky.... the general face.......... Even the cracks around the hands feels like it's lacking impact and...........
Yeah, the more I think about it, this just doesn't look great at all. I'm not sure we'll ever get this up to the standard of actual animation, but it's better than nothing, I guess. I just hope big corporations don't see this and just use something that looks this rough to replace actual artists.
I am really happy to see you using an artist who consented for the reference. That makes a big difference. Keep trying to be great and ethical at the same time.
I think if the voices can line up with the mouth movements, it will be solid gold
i feel like a lot of the detail and shading was lost with the new style as there was less training to go off of and it was a different style from the first one, but I think it also looked much more consistent! maybe traingin on more open-mouthed poses for the new one could have worked though, the comparative lack of training data did sort of show. Though to be fair even with highly trained ai videos it still gets mouths wrong sometimes, maybe in the future there can be an automatic sort of process where another ai can detect things like open mouths and map it to be used for the ai to match the footage more accurately. amazing to see how much better ai generation over video has gotten over such a short period of time!
I'm right there with you guys in finding new ways to use these AI tools in creative ways,
it's exiting to imagine what will be possible in just a few years, AI opens the door to a lot of creative possibilities
I'm still mixed on the use of AI art, mostly because of the training data being stolen artworks; but with this setup of yours, creating your own data set with an artist who consented, it gives me ease.
This is really great and a good example of how more traditional forms of art creation and the use of AI don't need to be adversarial.
Im glad to see you understand the use of AI. It is not about replacing anyone, it's about working hand by hand with the AI to make things e
"easier".
Is it possible to train AI with 3 D models of the characters in your anime? Maybe that will help with the flicker. It could use the models for the motion blur and perspective changes which may be causing the flickering. SN: I have zero effect experience btw I’m a photographer 😂
10:51 That's what I think is the problem right there.
The issue with Ai isn't artists like you making new and exciting content for the sake of creativity, I do fully agree you're on to something entirely different, and in fact is something quite intriguing and awesome. However, sadly, artists like you aren't the ones who lead the world of entertainment. The concern I personally have (and the one I've heard other animators like me have) is that companies and studios will only see the last part of your statement "Here we are, with a full episode of an anime, made with 6 people over 4 months". That's what's scary, that because of a new form of Vfx, the side of animation is in danger of being seen as something worthless by corporations, all while animators are already being neglected and exploited as I'm sure you know.
So I think that's the fear- that this fantastic tool is going to be exploited by corporations by kicking out animators and artists in exchange of something more efficient. Leaving artists with almost nothing to fight back- no matter what we do AI is always going to be faster than traditionally drawing and more efficient- and that's all that corporations see, and that's all that gets out to the world, making it all that people will see devaluing artists even more so than they already are- so If I were to assume why the controversy exists, it's just that... we are scared that something wonderful you are making is going to be used to harm other artists, is nothing against you or Ai really, it's fear of loosing our livelihood. It isn't just that AI is stealing art and what a machine is doing is being referred to as "art" (which is demeaning and terrible because it carries no emotion or creativity, it simply blindly replicates what others do and blends it together in a big slush of an image)
Right now they are writers on strike, and what Disney is doing is trying to replace them with Ai. You, with a small team (relative to Disney) was able to improve Ai this much, imagine what a corporation with ridiculous amounts of money can do with Ai- So it isn't even an irrational fear anymore... I'm happy that you've used an artist to create your style from a new, and I'm happy you're excited about the progression of Vfx with the use of Ai- but this doesn't really make it any better, if anything you're kind of proving the point that if anything the art world is becoming even more competitive at best and becoming truly horrific at worst, so while this isn't anything to do with corridor, it is a situation which may bring consequences might be a lot bigger and dangerous than one might imagine. So are you responsible for any of the things that might happen? Not really, but by normalizing the use of Ai it's discrediting what artists like me or other animators can do.
So if anything, I'd just ask that anyone using Ai can really understand that the commercial use and normalization of it hurts artists, not because of you, but because of how corporations and the public might react to something like this.
That's what I'd like to say, personal opinion but based on what I've seen with the art world I'm in... I hope it helps shed some light in what I think is the other problem you didn't get to address here. But again, thanks for getting an artist to make an art style, if the entertainment world wasn't so focused on making money and cared about art, that would solve the issue, but we don't live in that wonderful imaginary world were artists are valued...
But that's what I think, regardless thank you for the video addressing the artists, I know it isn't for me but as an artist I appreciate the effort :) it was a good progress to witness and it was great to hear your point of view
What you guys made is art, un-debatably. You used a tool that complements your creative art form. With WORK. It’s the next steps down in human input that put it in the ‘needs to be debated philosophically’
Flickering/consistency has been a huge problem for me. Fine tuning models that map one person to one character is a really interesting solution and clearly it's made a huge difference. This entire workflow is just on another level. Well done demonstrating the best this technology has to offer.
creating your own dataset also fixes the whole stealing other people's art issue.
What's weird is because the animation is so much better now, you can tell the wigs and flappy mouth movements which were better hidden in the first.
@User Unfriendlyno because if you make your own art you took inspiration from other art so youre still stealing
@Technus bro taking inspiration isnt stealing. same thing with references. ai takes ppls art and puts it in their dataset 90% of the time without the artists knowledge or consent
@Technusnot really since humans are not comparable to ai and ai is not comparable to humans the way you people compare it like “ oh stalking others and taking their photos without their knowledge is fine you look at those same people through your screen without permission or consent why should we “
I love the approach you're taking with this. A pencil is only as good as the soul which wields it. Excited to see your progress on this.
I think you are onto something amazing, first congratulations to the team.
As feedback I think that less flicker might not be the perfect solution, because then some flickers might become more visible, it would be similar to reducing the frame rate of a fast paced animation.
Keep up the good work!
Amazing, but I honestly loved the first one more, I literally had never seen anything like it, all the flicker gives it a unique touch, almost like an aura of tension
IMHO, jank in these kinds of things adds that feel of human touch. I think the more they stabilize this, the more it will look like just straight up 3D, and the less special it feels. Though on the flip side, it will probably make things easier for them to stylize it even more (like add controlled jank like traditional animation techniques), having cleaner samples to work on.
they literally ripped off the art from the vampire hunter d: bloodlust and put it through an ai...o but its not stealing, its re-inventing =P
@J.M. Gillespie Books Vampire Hunter D is not Castlevania.
@Krievv ya sorry it was a long time since i saw the other video, i rewatched and its bloodlust, not castlveania
@Krievv but my point still remains reguardless which anime
This looks really good! I think the 2 pieces that are the most jarring afterwards are the mouth movements and the motions. Animated character movement is generally not bound by human anatomy, which gives this movie a distinct feel in how the characters move within their environments.
Wonder if you could create characters in something like blender, with all your own styles and such, and then have an animation set that just ran through all the poses, lip movements etc and then feed that into stable diffusion. Could be a much more automated proccess and you just do the animation set once and put in on any new character model
Good on you for not using existing art. Getting a real artist to make a style guide is the absolute best way to go. Big ups.
I whole heartedly agree. I'm not anti machine learning/AI, but I am concerned when there's the non consensual taking of preexisting artwork. Which was why I felt irked by the previous video on the subject was the use of Vampire Hunter D. This felt like Corridor crew really thought through the concerns of the previous video's reception and took that into awareness
Ya, I'm really glad they aren't stealing art anymore.
the style transfer for the environment was still trained using other peoples art and work and they won't be compensated.
@BigBoyGandalf thats not how style transfer work, they dont work the same way a stable diffusion does
@loud noises im interested then, where does style transfer get theyre references and training from ? genious question.
The fact that the key to embetter the animation was to avoid stealing artists was pretty unexpected and welcome !
Interesting to see that the emotion came through better on part one
They basically traded the general flickering for the mouth animation(which doesnt work now) if you get proper sheets of main characters with different mouths and expressions it should solve both problems even further. (Adding more angles would help too)
This is more in line with how I feel like these styles of Copying AI should be used, taking your own assets and using it as a tool to expand your capabilities.
Some people I know at university made their own ai short, but due to copyright rules and limited time/crew they had to find some none copyrighted art that was in itself very inconsistent. The result was a flickering mess. Using your own art for this is much better and I think that it will be less controversial when the artist agrees to it and you have the right to use the artwork.
It's really cool seeing how this tool is used. I loved the first RPS and I look forward to seeing this one. Honestly though, I'm not too big a fan on the art style of this one. The Vampire Hunter D style looked so good. I understand why y'all went with your own art style. I just wish it was more on the anime side instead of the cartoon side
It’s because of new court cases they could get sued for the vampire hunter d art used. (Atleast if they tried using it for commercial reasons)
What art used to teach it effects if it’s legal or not.
They didn't want people to think they were "stealing" from other works, that kinda stuff is a pretty hot topic rn
@A L The courts have yet to determine anything.
Well sucks for you I guess
@A L yeah, I get that. I just wish the art style was a bit more anime than cartoon. It still looks absolutely stunning
i like that corridor shows us that AI is a tool and not a cheat that can make a whole movie in minutes (i can assure you having tried it takes quite a bit of work)
Great improvement than the first one. Less flicker, more consistent. Might be the 'feature' of rotoscoped animation (Since we also R&D in the most economical, fast way to do rotoscope, yet still looks animated (and not 'filter' look). :D
Wow, you guys did it better than the last time.
Stick with the in-between and keep the keyframe constant (human work whenever possible), and it should be good for now.
There is no doubt that this is art made in collaboration with AI, but guided by real human artists who put in a lot of painstaking work. And it looks astonishing at times, but the disconnected dialogue just breaks it for me... You were so close! Next time?
I’m so happy this got a continuation.
My biggest issues with AI generated art always comes back to consent of the artist whose works are being used in the training model. Training a model off of you own art (or commissioned art in this case) is IMO the ethical way to go about it. I really appreciate that you guys put in the work to use these tools responsibly!
Artist do the same. Copying things they have seen. So don't cry just because a machine is making artists useless. Go with the time.
It's a non-issue. Artists look at other works for inspiration or reference all the time.
@MrYTThor although it seems the machine is doing the same thing humans are doing from your perspective, it's pretty interpretable if it is or not. Our human minds aren't as straight forward as the digital neural networks that consist the learning models. I have a Computer Science degree and one of the classes specifically focused on this last year, they made sure we knew the neurons in a neural network are an extremely dumbed down version of our "analog" neurons. And not only that, they're deterministic. Whereas the jury is still out on if we're deterministic. Either way, a lot of factors, and a lot of reasons to believe either way of thinking
@Purple Moss Clumpthere's a difference between inspiration and straight up copying a style, it's not a non-issue and I think people are valid for having concerns about it
I think this view is going to be a double-edged sword. In the end, the only entities who are going to have the resources to be able to generate enough data to feed into these image models are the big corporations' gate keeping the jobs artists want to protect.
Then, the technology is going to be prohibitive in cost and use rights for the public.
Further empowering the largest corporations in their dominance over media.
I think what would help validate the improvements faster to your pipeline is to create a 3D model that will allow you to build out training data much quicker (as you can much more easily mess with lighting/mouth movements etc). That way you can experiment with how much training data you can remove before it makes the result worse and figure out what to actually ask your artist to draw.
For cohesiveness in the lighting maybe that can be a second process after the fact? ie. play around with an initial pass with just flat imagery with 0 shading and after another transformation that applies the "lighting" to that. No idea though as I've not played around with Stable Diffusion but I do think splitting the task out into smaller chunks will let you have more control and finetuning of the models.
I want an episode 3 with improvement in facial expressions and motions of bodies.
Anyway.. I love your video.
Hope one day A.I. will be able to creat motion pictures from comic books with exact same art style and dimensions of bodies. Love to see some 🗣️ahm !! “Normal Comics” comes to life.. 😅😊
For now now, AI is still not much more than a very advanced tool. Much more imitative rather then generative. And needs a skilled artist and a vision to develop its full potential. And you guys really demonstrated that. What amazing things can be achieved with this, but also how much effort is still involved in the process.
You were one of the first AI obsessed guys I saw on the internet, if not the first. It's awesome to see your creative progress so closely, thank you for sharing it with us.
I love anime rock paper scissors and tho it’s controversial I’m so glad to see you embrace it and make more! Love your content guys! Definitely like the anime art style of the first over the second but it’s still great
When you create your own art stylebook specific to the project and train the AI to do that specifically for each character, I'd day there's no longer any problem at all. It's now performance capture with AI-assisted animation. 100% your own product.
You will be surprised how people will still find something negative to say
People will still find problems with it, I guarantee it. Because "real art" is more than just the style being your own, its the execution too. The moment you give up control to an AI on that level, it opens things up to a lot of backlash.
The stylebook inside the artists mind is still Vampire Hunter D, so in effect its the same thing but with a human middleman.
@Kyle Artists do not own their style, only their works.
It's not 100% their own. It's still using a base model like SD 1.4 or 2.1, with the anime style Josh made layered over it. If they weren't using the base models, there would be no way to make this look coherent without an impossible amount of art. For reference, LAION-5B, the dataset which Stable Diffusion's base models were trained on, has over 5 billion reference images. That being said, it's still a big improvement from their first project in terms of ethics, and Stable Diffusion is probably one of the more ethical AI models of any kind purely by virtue of being open source. It's a cool project and more interesting than just using AI to replace people's jobs.
5:32 yeah but an emotional reaction doesn't always mean it's good art. The discussion is based upon if it even is art, or to what degree it is art. The emotional response was not to the "art" itself, but solely to the tools and techniques and them being viewed as a possible erosion or a flippant viewpoint towards animation by trying to present the thing that you made as being something approaching or being on par with the things that we highly regard as animation, like the source material for example. And you even presented it as the thing that's going to replace animators or that made their lives easier, when it's kinda disrespectful to try to compare the two things. the fact that you equated it as being validation for the "art" being good kinda worries me that you've missed the point people were trying to make or don't really understand the discussion, there's like a smugness that you knew it was gonna be controversial and you did it for the controversy and are very happy with yourself but idk man, I'm just one man who loves animation, and of course I could just not comment, and be like "hey let him make what he wants to make" and I feel that but something deep is calling me to even just add a little bit of context. Because vfx≠animation. Theres a lot of overlap but theres some distinct differences. Animation is very dear to me and it feels like its something worth fighting for. I'm not trying to fight against you, but I'm trying to fight to help you understand.
It's exciting when something can lower the bar to entry so more people can create something that was out of their reach. Understanding the basics of animation is incredibly important, but if you don't have access then you'll never have a good foundation to start from.
Neeko didn't need a wig. His hair is already magical.
Cool stuff. If the faces and mouth animation gets better you cant tell the difference between a "real" animation and one generated with ai.
v2 looks gorgeous!
Something I'd be interested to see in v3 is incorporating more stillness.
Like, most of the time in anime, only the eyes will move, or the mouth, or the hair, or one arm. A whole body moving is pretty rare. And even then, they probably move less than the "still" shots of your actor. Or it's more deliberate-looking, or something.
I feel like you guys are going for this in the composition and acting. (Esp 10:04-10:06, which I love. And maybe also that shot of the guy in the window.)
It would be interesting to experiment with mimicking the way people used to animate, like, just eyes on a see-thru paper and then place it over a blank face.
Like, 9:33-9:36, you could def just paste a moving head on a still body.
Or you could try letting the AI animate, cut out the moving eyes, and just paste them on a still image.
That might end up being nightmare fuel. Or the intermingled full-rotoscope-y next to partly-still might be too jarring.
(I know you guys know how anime is made. I just couldn't find a way to articulate my suggestion without writing it all out.)
Corridor showing what can be done when you put effort into the process of making stuff with AI instead of “this is the future of animation, fire your VFX artists”. Truly good stuff
This reminds me, a little bit, of Delta State. It was an animated show in Canada back in 2005. It would be cool to look at how they achieved this kind of “real actors with a cartoon shader” look.
I think that doing the voice recording as your videoing would make the dialogue match better. Similar to how actors record their voice while face tracking for cgi
From what I gathered the AI in the 1st animation grabbed info from VHD and that style often uses squint eyes, closed eyes then animating them wide open in times of shock surprise etc, this is a key style while the AI couldn't allocate where these changes should accour, so even tho you could take care of the flicker fixing the face is a nightmare.
So if you are going to train something I guess a code needs to exist that covers opening and closing the eyes, because it pulls it all over the place at random and if you spend time fixing it frame by frame might as well make everything from scratch since you will get more clean results lightyears better in comparison and original style too.
I like the consistent style based on one artist and I imagine the results could be improved by accumulating more artworks (especially for face animation), but still the concept is proved.
I'd love to see you now have some little collaboration with Aaron Blaise regarding traditional animation and boosting the workflow with Machine Learning. I suppose it might be interesting to replace video footage with simplified drawings as the input - and in this way achieve some squashing, stretching and overexaggeration of the animation.
I'm glad that you have now used AI and fed it your own art. This is the most ethical way of how to use machine learning software that uses images. The controversy stems from using copyrighted art and was used without the permission of the artist. I for one am now all in for the support of this because you've reached out to an artist and the artist created art with the knowledge and intention to have their art be used on the software.
Artists who believe they made up their own style devoid of all the media/art that inspired them are narcissistic liars.
@Patriot 03 It is just called "inspiration". The artist even explained it in the video that he looked at a bunch of anime and added his own spin to the style. Were you not watching the video? AI doesn't have inspiration because it just straight up copies several artworks pixel by pixel.
@D0decadice this.👍
@Patriot 03 you dont know what you are talking about of course we know we sample each others work, workflows, and techniques.
@D0decadice That's literally NOT how the AI works, it doesn't have a database of pixels anywhere that it uses during the generation process. Just like the artist the only reference is the inspiration gleaned from learned principles during the training process.
I do admit I love your passion and hype for this project.
The ONLY thing that worries me so so much about this, is whenever the big studios get their hands on this, jobs for animators will be cut in half for sure.
And jobs for animators are already barely a thing anymore, let alone jobs that pays well.
BUT, maybe the introduction of this will allow traditional animation to become a rarity, thus being more sought out? I dunno.
It's a scary world.
However, if we look at this project secluded from traditional animation, I think it's a pretty cool project where you evolve techniques, push limits, and try something new.
Unfortunately usually tech can't be held back by the possibility of lost jobs-- it just doesn't happen. The tool exists now, and the industry and those within it will almost certainly need to adapt to it. At least historically, a tool hasn't been stopped from being used at the risk of jobs being lost.
@Mobius Studios Crazy to think that 2d animation in the future will not be made by actual animators apparently.
As much as I agree with you, there is no denying that AI will eventually become the norm.
Right now there's a lot of very strong opposition towards it, and I understand it very much as an art student. However, history has shown many times that technology ALWAYS triumphs.
When digital art appeared, the traditional artists who used pen and papers also opposed it. Now Digital art is the norm. AI Art is a similar debate, IF we disregard the ethical aspect regarding the AI sampling on other artists without their consent (which is not a neglectable point).
On a practical standpoint, we will have no choice but adapt. Marc Brunet said he would welcome it as a tool, as long as it is not used as the finality, and I agree on with him. Use the AI to help yourself, but you still gotta use your artistic skills to polish the final result.
On a moral standpoint, I think our only hope is to have legal regulations on crediting sources / sampling consenting artists.
Look at it like this, if a company purely relies on these kind of programs without real talent to make it work just for the movie's story being bad in the end, then they weren't a good company to begin with. Meanwhile you have people like those in Corridor who are actually managing to pioneer this technology as the tool it was meant to be. The fact that they are willing to learn from the previous in order to improve goes to show that with enough drive and creativity, anyone can finally be able to see their ideas become reality without the need for big corporations who only ever cared about the bottom line. Now they just need to figure out the weird inconsistencies like with the eyes and etc. and there will be no telling where this will go.
@Kenddamus The analogy that AI art is anything close to traditional artist opposing digital artist is stupid.
sure, was there some pushback, but ultimately the basic need for skilled labor was still there. a digital Artist can still pull of great artwork on Paper, maybe the will struggle a little bit more, because they have to draw without all the quality of life tools digital art gave them, but AI art is not a quality of life tool right now.
it literally replaces artists. that's it.
And if the future of humanity is going to be that art and culture is AI controlled, then we as a species have no future.
Even though I'm still not on board with AI being used this way (Sorry, but nothing will beat animation by hand), I will say that I'm glad y'all at least hired an animator this time around and the output is *slightly* better than the first. But that's a very small *slightly*.
But my problem this time around is that this video just feels very defensive about this whole thing. I would honestly respect you guys more if you just came out and said 'We're gonna keep doing this, go jump off a cliff if you hate it so much'. Because that's the vibe I get, especially with that schpeel on art generating an emotional response. If anything you guys hiring an animator just goes to show that things look better when they have even the smallest human touch.
Cheers for the video guys! Some really interesting topics you bring up. It got me thinking on a couple of points too...
I would say that A.I. art isnt just a matter of taste, its also a matter of politics. I saw someone over on twitter put it in an interesting way earlier 'Humans doing the hard work on minimum wage while the robots make poetry and paint is not the endgame I imagined'. 😄
So I'm all for these tools being used by a passionate creative, but do we have to ask ourselves what the knock on effect is going to be? Is this technology going to open the door to new creators and allow them to make a real living, or is it just going to play into the hands of studio execs who will be able to produce anything they want on a shoestring and crowd everyone else out of the market?
I think theres more to the 'controversy' than corridor have touched on here too. I mean, it's a little bit in poor taste to make a video like this during the SAG-AFTRA strikes and not mention the concerns of all the artists out there whos livelihood is threatened by this tech. Are we kidding ourselves here thinking that this sort of thing isn't going to put people out of work? Its fairly obvious that the big studio execs are drooling at the thought of replacing all those 'uppity creatives' with a digital counterpart that they dont have to pay a living wage. What is the legislation around all this going to look like? Who is going to end up with the power of this content creation in their hands? Better to ask these questions now than to stumble over them in the future maybe.
Otherwise, I can see how this tech could be used to augment a piece of content, but I can also see how it could be used to turn it into a bland 'designed by committee' product too. Its always going to vary from project to project I suppose, but I think its worth considering as we develop the culture around this tech and how we use it. Are we going to stifle the potential of performers to practice their art? Is there a point where streamlining a creative process goes too far?
And by that I mean, are we going to end up depriving ourselves of the joy in the process of creation in the end? What film fan doesnt look back at some of the analogue techniques and think that we've kinda lost something along the way for example? It was probably a pain in the ass back in the day too, but I still think its worth considering haha.
Personally, I'm a bit tired of the movie scene out there at the moment. So many Marvelesque movies that are just being pumped out and it all just feels so bland. And I say that as a nerd who would have been excited for all that stuff in my time! It just worries me that we're going to see that same 'safe', moneymaking approach applied to more and more mediums out there.
Hope I'm not being too critical. I very much appreciate the effort you guys put in for this and I can see the huge potential here too. I just think we need to plot our course wisely so that the maximum amount of people see the benefit from it all. Its about being smart and nuanced maybe.
Speaking of nuance, I think a great test of this tech might be to see if you guys can truly replicate (or even enhance) the "emotional connection" garnered from a tried and tested peice of art. If you guys were to take say, Boromirs death scene in LOTR for example and truly capture the subtlties there, I think you might have something to write home about. Make a room full of grown ass people cry with this stuff and you've got my attention. 😂
It's cool to think of the AI as tool that can reproduce a style. It's more like a student to a real artist, rather than a replacement. Perhaps in the future, artists will have copyright over their original works, and then can either choose to license or reserve the rights to their art style.
I definitely think this is a step in the right direction for you guys, as far as AI goes.
I don't necessarily like AI, but I think the fact that it's now trained on an artist who's given consent is fantastic.
With the visuals, I think the lip flaps still aren't that great, and there is still a bit of inconsistency here and there.
I wonder, what would it be like if you picked and chose specific frames to keep, and had your artist manually tweak them to touch it up a bit? It would be a bit more anime-like (having fewer frames, but animated more strategically), and it keeps a larger human element (not to discredit all your other work on the project, of course), while still automating the process a bit.
Awesome Job guys! It's obvious that a lot of thought, effort, and artistic technique goes into everything you do down to the wardrobe - Just make your art and have fun the way you do - If people don't like it, they can watch something else - Most of us, just want to be entertained, and your content does that perfectly, so keep up that great work :)
love your channel!