Keeping up to date with AI trends as I always try to do, I watched two update videos by Curious Refuge.
First they talk about many text to video AI updates, including a new stable video diffusion feature which looks promising, but after a comparison with Pika 1.0 and Runway, it becomes obvious that it is still in earlier stages of development and that a mixture of the three tools will still probably be the best workflow.
They continue to talk about some Elevenlabs uptades that I have actually already tried out for the Anime AI film trailer and about a paper on a gaussian splatting algorithm that supports animation and physics simulations.
In the next update video, Caleb Ward talks about the new Pika 1.0 in more detail. In a showcase, it seems that the video-to-video features are in their infancy at this point. Changing backgrounds in videos and expanding videos (much like photosho’s generative fill) seem to work almost perfectly, however. Adding VFX simulations shows heavily varying results. He does not go into any detail regarding text-to-video functionalities, much less animation or stylised animations, which is a real shame since that’s basically what I want to be doing for my thesis.
Next, the video talks about Gemini, Gemini Pro and Ultra, upcoming Chat-GPT competitors by Google which supposedly beat Chat-GPT in 30 of 32 tests. Ward thinks that OpenAI has nothing to worry about, given the recent news of the company’s Q* project supposedly being too powerful, concerning the company’s investors.
MagicAnimate and Animate Anyone are new AI powered animation tools that can take images and essentially add motion from a reference video to that image. What’s particularly interesting for me here, is that apart from the many TikTok dancers, there is a brief showcase of an anime figure being animated very nicely.
There is also a tech demo on real-time conversion of real footage into anime using Stable Fast, the results are extremely rough at the moment, but the fact that the technology works in real time is promising to say the least and considering the rate at which these technologies are progressing, maybe it could be ready for actual use in time for my master’s thesis.
