2024 STATE OF THE VFX/ANIMATION INDUSTRY: FULL SPEED AHEAD

By CHRIS McGOWAN

Technological advances in the VFX/animation industry don’t look to slow down anytime soon, despite difficult-to-predict economic adjustments up or down in 2024. “We saw huge growth pre-2020 with a decade of Marvel and large-scale VFX productions,” according to Gaurav Gupta, FutureWorks CEO. Then came the rise of streaming services and a post-pandemic boom in content. “The arms race for content is real and has had an impact on the VFX industry too,” Gupta adds. However, with the strikes halting productions, “There was a contraction in this expansion,” Gupta notes. There was also belt-tightening among studios and streamers looking for greater profits. Irrespective of lulls and/or recoveries in film and TV production, the VFX/animation industry should continue to dazzle in 2024 with new developments in tools and techniques. Real-time technology, LED volumes, virtual production, facial recognition, volumetric capture, technical collaborations like OpenUSD and other innovations continue to move the craft forward. In addition, globalization, outsourcing, collaboration and remote workflows are also buoying the industry. And AI, nearly everyone’s favorite topic for debate, promises new efficiencies and creative possibilities for visual effects. Here are some thoughts from top executives about trends, breakthroughs and the business.

Grant Miller, Partner & Executive VFX Supervisor, Ingenuity Studios

AI really pushed into the mainstream VFX space [in 2023]. While there’s been apprehension around how the technology will be used, we’ve seen a lot of positives so far. For example, clients are using Midjourney to clearly articulate their vision. Software, like Cascadeur, is using AI to add believable secondary and physics to animations. AI-generated Neural Radiance Fields (NeRFs) are exploring novel ways to represent 3D scenes.

While we’re not quite there yet, AI also has the potential to remove many tedious tasks in the pipeline, affording artists more time to focus on quality and creativity. It will also enable ideas and workflows that would be nearly impossible without the help of machine learning. In the near future, we’ll be able to create robust depth and segmentation maps for moving footage, allowing artists to add fog to a forest or change the leaves to autumn.

The industry has shifted from chemical compositing to digital, from stop-motion to computer animation and from matte paintings on glass to fully 3D set extensions. Technology has driven each of these transitions, and each has increased the use of VFX in the content we consume. We’re excited about what the future will hold as AI improves the tools we use in our craft, and I’m thankful to be working in the industry during such a transformative time.

Fiona Walkinshaw, CEO, Film & Episodic, Framestore

AI and machine learning are obviously topics that have everyone talking at the moment – in reality, in our industry we have been exploring the use of AI/ML for quite some time. We will continue to explore their use and application and how they might speed up compute time and processes that enable our artists to do what they do best: creative immersive worlds and believable CG characters and creatures that require human artistry and experience to achieve.

I think we’ll see the continued expansion of FPS, our pre-pro- duction services division as more and more directors, showrunners and execs get to grips with how previs, techvis, virtual production tools and postvis can help them plan their shows and achieve their vision. Alongside this, we’re constantly improving and enhancing our in-house tech, like FUSE and our work with Unreal and FarSight, our proprietary VP ecosystem which encompasses virtual scouting, motion capture, virtual camera and on-set visualization. All of this speaks to the same broad aim: to deliver the absolute best in VFX and animation, breathe life into everything we work on and reframe what directors and audiences alike can expect from the stories they see on screen.

Jennie Zeiher, President, Rising Sun Pictures

Early in 2023, we started seeing hype around Generative AI. The “sudden emergence” of Artificial Intelligence (I prefer machine learning) was years, decades in the making. Gen AI created wide- spread mania, as many of us were not prepared for it. VFX started to see new applications being promoted, from concept apps such as Midjourney and Stable Diffusion, through to animation, including Wonder Dynamics’ Wonder Studio that animates and composites a library of bipedal CG characters into live-action material. These applications allow us to dream of what can be achieved next using machine learning. We should see ML as adding more tools into our VFX toolkit and amplifying jobs; pushing the boundaries of what we can achieve in VFX with more complex challenges.

The sudden emergence has been building slowly over time. The term “machine learning” was first popularized in 1959 by [computer scientist] Arthur Samuel, who created one of the first self-learning programs, a simple game of Checkers.

RSP has been working away in ML for the past five years. We have made huge inroads in research and development, as well as integrating our ML team into production. Our REVIZE toolkit is evolving, having started with ML applications around face swapping and manipulation, through to the body and now performance. It is RSP’s mission to integrate our traditional VFX pipeline using ML to address our client’s technical challenges, alongside the know-how of experienced VFX creative and technical teams, all while producing shots at scale. I look forward to what 2024 will bring in this space.

Hitesh Shah, CEO and Founder, BOT VFX

Fundamental VFX demand still looks strong as it has for the last few years based on three trends: (1) continuing growth in the amount of content produced, (2) continuing growth of the share of overall content needing VFX services (i.e. a larger percentage of shots need some digital effects), and (3) pent-up demand from the 2023 strikes.

While the streaming platform strategies evolve relative to subscriber growth versus profitability and paid-only versus ad-supported, generally speaking, the amount of content being produced is still fairly strong in the foreseeable future.

[Key trend for 2024]: Greater application of “human in the loop” AI tools in VFX production processes.

Joanne Smithies, Visualization Supervisor, The Third Floor

Many would mention the continued advancement of real-time rendering technology and rightly so. However, as a previs supervisor, I have to highlight the advancement of virtual production tools. These tools are not only becoming more powerful but also more intuitive and portable. Directors, DoPs and VFX supervisors can now bring the CG elements of their film on-location scout or on set, giving them more creative control by enabling them to plan and visualize their shots with all the elements in hand.

Addy Ghani, Vice President of Virtual Production, disguise

In the next 12 months, virtual production workflows will become more streamlined and simplified than ever before. The stages themselves will be more flexible – that is, they will be easier to dismantle and rebuild in various shapes and sizes as the shot requires.

There’ll also be an increase in content flexibility – creatives will be able to choose exactly what type of content to display on their LED volumes, based on parameters like their budget and scheduling restrictions. This can come from many sources: video plates to 2.5D (multiple video plates), AI generated content such as NeRFs, or real-time rendered 3D environments. For the most elaborate and demanding productions, fully realized 3D real-time environments will be built and customized on the fly. For more budget restricted shots and sequences, video plates, 2.5D and AI generated plates will be used frequently.

For VFX artists, that means there will be more flexibility to use virtual production technology on a much wider variety of projects, whether that’s a Hollywood feature or an indie short.

Douglas Green, Co-Founder and COO, DI4D

Real-time game engines have broadened the creative possibilities available to directors. Not least, in the production of high-fidelity, realistic animated films such as those produced for the Netflix series Love, Death + Robots. The challenge that faces directors and studios is matching the visual quality that is now possible with equally realistic character animation. Performance capture meets this challenge head-on, driving facial and body movements with the skill and craft of a real actor’s performance.

Through the use of digital doubles, characters can remain faithful to the exact likeness and performance of an actor. With the digital-double approach, it’s possible to create the immersion of cinema or theatre entirely in CG, all while preserving the artform of acting. Because each performance is always unique, it ensures a greater level of authenticity and audience engagement, which will be vital to studios and filmmakers in 2024.

Lala Gavgavian, President & COO, Digital Domain

[Digital Domain’s] journey has led us to create multiple proprietary tools to support and streamline the filmmaking process. These innovative tools and processes have propelled efficiency in various components of our pipeline. We strategically integrate cutting- edge AI and ML tools, allowing talent to spend more time on the creative and technological components toward the final pixel of a shot. This collaborative ecosystem fosters growth, enriches our business and increases productivity. With our unique resources, we drive progress, support collaboration, and unify teams across global geographic locations, advancing the company and industry.

Eamonn Butler, Head of Animation, Cinesite

Recently, the combination of 2D and 3D animation has become one of the most popular animation trends; this will continue in 2024. We’ve seen this at Cinesite with 2D and 3D being combined in Space Jam: A New Legacy, Teenage Mutant Ninja Turtles: Mutant Mayhem and the upcoming Smurfs movie. Blending a variety of artistic mediums seamlessly into one animated video is a steadily increasing trend and allows us to reflect on how different styles of animation can challenge and complement each other creatively to produce innovative animation trends in the future.

Animation has been unable to escape the era of the revival, with nostalgic and retro animation creeping their way into popular animation trends in recent years. When working on Teenage Mutant Ninja Turtles: Mutant Mayhem, we applied film grain, noise, lens flares and dust on the lens to a lot of our shots and sequences. A perfected blend of retro-futuristic visuals and a collective longing for yesteryear will almost certainly maintain retro animation as one of the most popular animation trends for 2024.

[Regarding NPR], most of the filmmakers we work with are looking for something “different” with regard to the look of CG-animated films. There’s been a huge interest in projects utilizing artistic looks with films such as TMNT and Spider-Verse. It’s an exciting time for artists looking to push the boundaries and capabilities of CGI.

AI will continue to be a hot topic in 2024. In broad terms, we can say that although innovation is happening at a breakneck speed in these areas and now more than ever, it is difficult to predict what the long-term future will look like. I think it is fair to say that for the foreseeable future, high-end filmmakers will continue to depend on the services like the ones Cinesite provides to produce high- quality content. On the very low end of the spectrum (YouTube, TikTok, Twitter, etc.), generative AI tools will make VFX and animation accessible to a much broader audience, but at a quality bar that will not be suited for our clients.

At the moment, there are a few issues around using AI. In particular, there is the potential for disputes over rights and ownership. This will need to get ironed out before any studio takes a risk with tools that effectively mimic other artworks. I suspect that AI technology will take many years of development before it becomes a cheaper and better alternative to current production techniques. I believe you will always need artists to drive it, and it will only be as good as the vision set by a director.

Gaurav Gupta, CEO, FutureWorks

We together need to create better working conditions and a more sustainable model for both artists and studios. I feel it’s time not just to look at growth but understand how the VFX industry can work more collaboratively to achieve a healthier and more profitable way of working.

New tools and technology like AI will only help us to create more compelling visuals. Whether in terms of creating visuals or writing code, it will simply democratize and offer greater access to technology. This creates a more competitive and healthier environment. 2024 will be a very exciting year for the use of AI across all of the content production chain. It impacts everything from script to visual ideation, to VFX production, editing, sound. We’re going to see AI being used everywhere. This is going to be an exciting shift that will democratize tools and access to storytelling, as well as creating a healthier and more competitive environment for content creators at all levels.

It will also be a big year for virtual production – just look at how this has developed since 2020 with The Mandalorian. 2024 will see the adoption at a much larger scale as the entire ecosystem from camera manufacturers and other technology purveyors like Epic and Unity have made virtual production more accessible. Another exciting new development will be the launch of Apple’s Vision Pro. I can’t wait to see how storytelling and new content experiences evolve in this new spatial computing format. The wealth of content will only continue to grow.

Rob Hifle, CEO and Creative Director, Lux Aeterna VFX

2023 [was] an incredibly exciting time in the VFX industry. At Lux Aeterna, we have seen continued demand for unique visual experiences in documentary, drama and episodic, requiring high-end environmental and abstract simulations.

The most significant VFX industry trend in 2023 [was] the increasing integration of artificial intelligence and machine learning into the creative process. This year, we received a grant from Digital Catapult to pursue R&D in AI toolsets for VFX pipelines with the support of NVIDIA. AI-powered tools are revolutionizing how we work with regard to our VFX pipelines. There will be a huge surge in usage demand for these technological advancements as commercially licensable options become available.

The latest advancements in technology are opening up new creative possibilities. VFX artists are exploring novel aesthetic directions and experimental approaches as tools and techniques evolve. These unique approaches are challenging traditional pipelines, and 2024 will be an exciting time in reapproaching how we work with clients. VFX studios will be more involved and integrated at the inception stage, using their insights into new technologies and ways of working in order to maximize ambition and reduce risk.

Technological advancements have been progressing at a phenomenal rate. The adoption of real-time technologies like Unreal Engine for virtual production applications and beyond is something we expect to see continue into 2024. Despite some of the challenges faced in this area, both the offerings available and potential rewards for studios are very enticing. This accessibility and democratization of the tools has hugely contributed to the wide-spread adoption of these techniques, and I can only see this expanding into 2024.

Continued advancements in real-time rendering technology will lead to a significant shift in how VFX are created and integrated into projects. Real-time rendering will become more prevalent for both previsualization and final output, reducing the need for lengthy rendering times. These more sustainable practices will have significant reductions in energy costs leading to a greener VFX industry.

Neishaw Ali, Founding Partner, President and Executive Producer, Spin VFX

As the ‘hybrid’ work environment settles in 2024, more studios will either keep their current office footprint or reduce it. Therefore, instead of purchasing more computers, they may look to spinning up workstations in the cloud when they need it, as they can match the specific needs of the department and production without the delivery lead times and work with the latest and greatest systems. Cloud render – this service will be widely used, but is still costly.

The hope is that more providers will come online and provide competitive pricing, which will allow studios to utilize the cloud render to service the work on a “just in time” basis. Cloud storage is going to become even more important as studios work with teams around the world in this hybrid environment. The need for speed in data accessibility whether online or offline could make a difference in delivery of a show.

[Regarding AI/ML,] VFX studios are looking at smart ways to become more efficient, and I believe that they will look at automating 70% to 80% of certain tasks, such as tracking and roto using commercially available machine learning tools. This will be a great benefit to artists as they don’t need to spend time on such tedious work, but can now focus on refining and beautifying the shot. Advancements in machine learning-based denoising will allow for faster and cheaper rendering with no loss in quality. This technology has been evolving for years but is now at the point where the quality and fine detail are easily preserved.

Machine learning-based upressing algorithms are opening up the possibility of rendering at lower resolutions and then seamlessly upressing the imagery to increase speed and efficiency.

Generative AI has the potential to be a powerful creative tool for helping artists and designers quickly create high-quality concept art. What the future holds with technology is difficult to predict, but it will certainly be fast-moving and consequential.

Real-time rendering – the breakthroughs in real-time raytracing continue. Hardware and software have matured to the point where the large datasets required for some photoreal VFX work can be raytraced in real-time on a single GPU.

Michael Ford, CTO, Sony Pictures Imageworks

In terms of the state of the craft, it’s an amazing time to be in VFX and animation. The creative aspirations of the projects are increasingly ambitious. Filmmakers and artists are collaborating to create remarkable films. Audiences are responding, as evidenced in both the box office and streaming success of VFX-driven and animated content.

On the technology side, we have seen advances in filmmaking and collaboration across the industry. Open source technologies continue to help shape a future where both studio and software vendors can depend on open standards that work easily and effectively across software platforms and studio pipelines. With USD, Material X, Open Color IO, Open Shading Language and countless other tools, we are moving closer and closer to having a unified toolkit of fundamental building blocks that will benefit the entire VFX and animation ecosystem. There are also a lot of exciting developments in the area of the Open Review Initiative led by Autodesk, DNEG and our team at Imageworks. The idea of creating an open source platform for reviewing media and material for our industry is a significant development that would allow creatives to see their amazing work across compatible viewing platforms. The Academy Software Foundation (ASWF) and the newly-formed Alliance for Open USD (AOUSD) are leading these important and industry changing developments. It is an exciting time.

The impact of AI and machine learning will also continue to play a significant role in impacting the filmmaking process. With the rate of change in this area of technology at its highest level ever, we are going to start seeing studios implementing toolsets that allow artists to accelerate workflows and step function certain areas of the production pipeline.

Steve May, Chief Technology Officer, Pixar Animation Studios and Chairperson of the Alliance for OpenUSD

Universal Scene Description (USD) was invented at Pixar and is the technological foundation of our state-of-the-art animation pipeline. OpenUSD is based on years of research and application in Pixar filmmaking. We open-sourced the project in 2016, and the influence of OpenUSD now expands beyond film, visual effects and animation and into other industries that increasingly rely on 3D data for media interchange.

USD stands for Universal Scene Description, and Pixar created USD to make animated movies that have incredibly complex 3D scenes comprised of millions or even billions of individual objects. Creating such scenes involves complex artist workflows and many 3D content software tools. Historically, those 3D tools all used different data and file formats.

Pixar wanted to enable more powerful creative expression for its artists by streamlining their workflows and allowing the same data to be interchanged by all of the content creation tools. So, in 2016, Pixar decided to share USD and make it open source, allowing it to be incorporated into software across the industry.

Fast forward to today and USD now serves to simplify working on 3D assets across multiple technology platforms. Where this used to be very complex, USD makes it simple.

Recently, Pixar, alongside NVIDIA, Apple, Adobe and Autodesk, announced the Alliance for OpenUSD (AOUSD) (www.aousd.org). Many companies consider a standard for 3D data and content critical to their business operations and growth, and AOUSD is open for new members to join.

AOUSD is a big deal. We are on a journey with the evolution of 3D technology and OpenUSD is at the heart of it. If we’re successful, it will become the fundamental building block on which all 3D content will be created. Whether it’s immersive 3D content, new spatial computing platforms, or scientific and industrial applications – they will all rely on OpenUSD, and the Alliance for OpenUSD is the next step on the path to realizing that potential.