2023 GLOBAL PERSPECTIVE: THOUGHT LEADERS ON THE VFX AND ANIMATION SUPERHIGHWAY

By JIM McCULLAUGH

Iwájú, a comics-style animated series set in a futuristic Lagos, Nigeria, is the first production between Disney Animation and an outside studio, African animation company Kugali. CG animation is provided by Cinesite. (Image courtesy of Disney+)

What does the road ahead look like for the VFX and animation industries? The subjects that are most top of mind for those at the nexus of VFX and animation include: real-time, virtual production, LED volumes, AI, machine learning, AR, the Cloud, hybrid working, tangible effects of the pandemic, global expansion and the search for talent. With the world canvas now a bullet train of VFX-infused movies, via streaming and related platforms, a global cross-section of industry leaders meets in this VFX Voice virtual roundtable to discuss the outlook for the new year.

Paul Salvini, Global Chief Technology Officer, DNEG

When I think of global trends happening in the VFX community, the one that excites me most at DNEG is how real-time technology is establishing itself within our film pipeline. With the determination to provide our artists with the best possible content creation tools, DNEG’s UX (User Experience) and R&D teams have been working closely with our artists to find innovative and better ways of working by leveraging the power and immediate feedback of real-time technologies.

Over the last year, we completed several successful projects using our new hybrid real-time pipeline: an animated short film (“Mr. Spam Gets a New Hat”), final pixel environments for a major feature film (The Matrix Resurrections), and digital backgrounds for various virtual production projects. Thanks to the immediate feedback that real-time technologies provide, artists have more time to iterate and explore creative possibilities. The results speak for themselves. The quality of real-time output today is impressive.

Johnny Fisk, President, FuseFX

This new renaissance of entertainment touches every one of us. We’ve seen our industry explode in all directions as the use
of VFX continues to proliferate throughout all aspects of the market. High-end content is now being produced for all platforms in media. With the growing work, we, too, are stepping into the next generation of VFX. We’ve only just begun scratching the surface of how emerging tools and techniques can be utilized to tell bigger and more engaging stories moving forward. As artists, we’re forging new territory, such as utilizing real-time software and deep learning technology in our imagery and workflows. Bringing innovation to the table is bringing a renewed energy to all of our work. I don’t think there’s a more exciting time to be working in VFX than right now.

“The most recent example of a game-changer for us was the way our artists leveraged our latest AI face-swapping tools as an element to create the youthful Skywalker in The Book of Boba Fett. Combining the best of our digital facial animation techniques with the latest in machine learning really lets us achieve a combination of likeness and detail that wouldn’t have been possible just a few years ago.”
—Rob Bredow, Senior Vice President & Chief Creative Officer, ILM

Mission Impossible: Dead Reckoning – Part One is the seventh entry in the series. Visual effects and animation are supplied by ILM London.

Shazam! Fury of the Gods follows in the wake of 2019’s Shazam! SFX vendors include DNEG, OPSIS, RISE Visual Effects Studios, Scanline VFX and Weta Digital. (Image courtesy of Warner Bros. Pictures)

Christopher Edwards, Founder & CEO, The Third Floor

Regarding location-based entertainment, experiential entertainment and the real promise of the Metaverse… At The Third Floor we have always been storytellers at heart. We love helping visionaries take audiences on visceral, emotional journeys, but this isn’t strictly limited to linear media. Modern audiences are increasingly obsessed with quality interactive and immersive experiences that can take their sense of engagement to the next level.

For over 18 years, The Third Floor team has crafted cinematic moments for AAA video games and world-class theme parks, including Super Nintendo World at Universal Studios Japan. The pandemic sequestered so many people for so long that there has been a societal shift towards appreciating communal events. Whether this is a physical gathering, such as a live concert or a trip to an amusement park, or a virtual gathering in an MMO (Massive Multiplayer Online) game, there is nothing quite as satisfying as a shared experience with friends and family. So, content creators are beginning to adapt and expand their IPs to formats that complement traditional media formats and encourage social engagement and viral marketing.

Tram Le-Jones, Vice President of Solutions, ftrack

The pandemic has encouraged a lot of new thinking, which is very exciting for us as an industry. With the pivot to working from home, we’ve realized that making changes wasn’t as hard as we thought. The pandemic disrupted our norms, made us realize what’s really important in our lives, and forced us to do things personally and professionally we hadn’t done before. We’re making new connections that have opened us to adjacent and nonadjacent industries. It isn’t an entirely new concept, but the pandemic has accelerated it. Not only are we learning from others, but also they are learning from us. We’re working more collaboratively and finding that we all have much more in common than we originally thought. We’re going to see a lot more from this intersection.

“Things that would have taken months of cooking in the animation process can now take seconds. We can switch style sheets instantly, and a machine can re-code entire projects instantly as opposed to hours of hand, frame-by-frame labor. AI is an amazing way to start a conversation or to drive inspiration.”
—Andrew Melchior, Executive Vice President/Global Director of Brand Experience, The Mill

Mario (voiced by Chris Pratt) travels through an underground labyrinth in The Super Mario Bros. Movie. Illumination Studios Paris provide character animation and computer graphics. (Image courtesy of Universal Pictures)

The Little Mermaid goes live-action. VFX/SFX vendors include Framestore, ILM, DNEG, Rodeo FX, MPC, Lifecast and Clear Angle Studios. (Image courtesy of Walt Disney Studios)

Kim Libreri, Chief Technology Officer, Epic Games

Real-time technology will continue to have significant impacts on filmmaking and entertainment. As we’ve seen with the explosion of virtual production and in-camera visual effects in particular, VFX crews are becoming more and more of an integral part of the primary on-set filmmaking process. VFX artists are now joining the ranks of cinematographers, production designers, costume designers and other roles that shape production from its earliest stages. As the VFX process becomes more immediate and tactile to key creatives, artists are collaborating and iterating more with other departments. This dynamic reduces miscommunication and repetition, as creative decisions can be made interactively while in production.

Furthermore, emerging tools and workflows are starting to make transmedia production a reality. With Unreal Engine, for example, you only need to create your content once, and then you can easily deploy it across film, games, immersive experiences and other forms of art and multimedia. Real-time production is making it easier than ever to completely rethink how and where your IP can be consumed. As filmmakers become more comfortable with this new reality, adapting game content for film, and vice versa, will become the norm.

Michael Ford, Chief Technology Officer, Sony Pictures Imageworks

I’m incredibly excited about the continued industry adoption and participation in open source software initiatives. With the leadership and structure provided by the Academy Software Foundation (ASWF), the VFX and animation industry is making great strides to work together as a community to build software, libraries and processes that benefit us all. At Imageworks, we like to say that open source is the “engine of innovation” that allows us to leverage not just our talents, but also the talents of an entire industry. Open source also allows us to reach a more diverse group of people that might otherwise not have the opportunity to work in our industry, and we need this more than ever in order to build and strengthen our global workforce.

With the expanding use of game engines and faster compute via GPUs and distributed CPU rendering, the industry is moving towards a real-time future where creative decision-making is being made at a much higher rate. ICVFX (in-camera visual effects), animation and VFX workflows are all being influenced by these enabling technologies. I think the next few years are really going to change the way we think about computer graphics, especially when we look to the future of generating new and innovative looks via AI and machine learning.

Mathieu Boucher, Vice President of Operations, Hybride/Ubisoft

In the last few years, remote work has proved to be very efficient for many studios in the VFX industry, including Hybride. It opened the door to new possibilities and access to a broader and more diverse talent pool. For instance, since 2020, Hybride has managed to significantly grow its workforce with team members working from all over Quebec. I am interested to see how the VFX industry will continue to adapt to this new reality.

In 2023, we will start to see a democratization of the virtual production pipeline. The technical complexity and extensive pre-productions are becoming easier to manage while industry expertise and know-how are increasing immensely. This will open new creative opportunities for a wider spectrum of productions. AI is increasingly part of our processes, but I think it will also lead to impactful technological evolutions.

Frank Montero, Managing Director/President, ROE Visual US, Inc.

The exploration of virtual production borrows from the sentiment ‘With great risk often comes great reward.’ While advancements are continuously taking place, a certain amount of ambiguity is naturally associated with this budding technology. In truth, the concept of virtual production in film is far from revolutionary; however, the ways in which it has expanded to include digital artwork and LED displays are. Today, VP techniques facilitate production in a myriad of departments throughout studios worldwide. Most prominently, the use of LED displays for backgrounds on set simplifies the work for VFX teams in post-production while increasing real-time engagement for the cast and crew. The dynamic nature of the digital content ensures production can move forward in the desired direction while on-set modifications to the LED canvas can take place at any point in the process.

Adrianna ‘AJ’ Cohen, Senior Vice President/Global Head of Production, Mikros Animation

The technology in animation is improving at an incredible rate. An animator’s ability to create uniquely beautiful animation is getting easier and more accessible. This allows all previously unattainable ideas to come to life (e.g., mocap, animation/live-action hybrids, 2D and 3D content, etc.), and studios can take more chances on ideas they couldn’t previously afford to.

In addition to the advancement in technology, the movement to streaming providers has created a demand never seen before. Everything is changing. The need for resources will drive opportunities for every artist across the globe, which is an excellent opportunity for Mikros Animation and everyone working in the field. The challenge as a studio, however, will be to figure out how to attract and train talent, and make them part of our family.

Wayne Brinton, Business Development Director, Rodeo FX

The consumers’ high standards are not just a question of visuals. It’s about getting the same emotional experience you had when you consumed the content on your screen the first time – no matter the platform or the format. The sheer amount of content consumption in the past few decades has created expectations of fidelity in visual effects. When they/we don’t get that in an experience (ads, movies or even Snapchat filters), the experience becomes less than what it “should have been.” Like trying to redo the dragons from Game of Thrones in a Snapchat filter. Of course, users are going to be disappointed.
Consumers are expecting a very high standard in terms of imagery and visuals, yes, but that’s not really what the expectation is rooted in – they expect great storytelling.

“The benefits of virtual production are driving the growth, with the number of LED volumes likely to double in the next three to four years. While the need for post-production will shrink, digital pre-production work will increase in order to optimize the on-set shoot. Novel uses for LED volumes beyond virtual production will also help drive the growth of LED volumes.”
—Kim Davidson, President & CEO, SideFX Software

The Oscar-winning Spider-Verse saga opens a new chapter with Spider-Man: Across the Spider Verse, Part One. Sony Pictures Imageworks provides imagery and animation. (Image courtesy of Columbia Pictures and Marvel Entertainment)

John Wick: Chapter 4 is the fourth installment of the one-man-takes-on the-entire-underworld series. Primary VFX vendor is The Yard VFX, with contributions from One of Us and Mavericks VFX. (Image courtesy of Lionsgate)

“Due to the increased demand for animated series, traditional animation will have to investigate leaner techniques, such as bypassing the 2D process altogether and jumping straight into 3D, using game engine technology to be able to scale creative output.”
—Mariana Acuña Acosta, Senior Vice President, Global Virtual Production, Technicolor

Dungeons & Dragons: Honor Among Thieves deploys the latest VFX to honor the franchise. ILM, MPC, Clear Angle Studios and Legacy Effects contribute special effects. (Image courtesy of Hasbro and Paramount Studios)

Trek across post-pandemic America with Joel and Ellie in the HBO Max TV series The Last of Us. Primary VFX/SFX vendors are DNEG and RISE Visual Effects Studios. (Image courtesy of HBO Max)

Dennis Kleyn, NVX, Founder/CEO/VFX Creative Director, Planet X

Virtual production is gaining noticeable traction in the Netherlands as well as in this part of Europe in general. Our Dutch film industry is relatively small and not on the most progressive/ innovative side, so it feels a bit like most producers have just caught up with considering ‘traditional VFX’ as a creative department within the filmmaking process (rather than a problem-fixing one), and now an even newer technique is on the doorstep. Planet X has been involved in the founding of the first VP/ICVFX studio in the Netherlands: ReadySet Studios.

Rob Bredow, Senior Vice President & Chief Creative Officer, ILM

We’re very fortunate at ILM to get to work with world-class innovative filmmakers and showrunners who push us to invent new techniques on nearly every new show. Just yesterday, I was on set on one of our shows with a talented director of photography who was inventing new workflows on that day, and seeing our StageCraft team respond with just the right artistic and technical solutions just in time to shoot. It was inspiring.

AI and machine learning techniques are transforming the way software is written – and the way our artists interact with our tools. The most recent example of a game-changer for us was the way our artists leveraged our latest AI face-swapping tools as an element to create the youthful Skywalker in The Book of Boba Fett. Combining the best of our digital facial animation techniques with the latest in machine learning really lets us achieve a combination of likeness and detail that wouldn’t have been possible just a few years ago.

Andrew Melchior, Executive Vice President/Global Director of Brand Experience, The Mill

AI is such a sophisticated and wild beast. There is always the question of whether the thing you’ve created will stay within the black boxes, and there is a lot of concern about that. One thing is for sure: AI certainly creates a stir and an interest.

Regarding AR, VR and AI and their relationship to the evolution of VFX/animation, from a visual effects point of view, we have been having lots of conversations. Things that would have taken months of cooking in the animation process can now take seconds.  We can switch style sheets instantly, and a machine can re-code entire projects instantly as opposed to hours of hand, frame-by-frame labor. AI is an amazing way to start a conversation or to drive inspiration.

We can address and scale down huge 3D geometries and make them real-time assets that can run on local devices. With NeRF [Neutral Radiance Fields] – instead of taking very detailed geometry and point clouds, you can now take 2D photos, and machine learning can take them and build 3D models on the fly. This means you can easily create characters with true-to-life shadows and textures that would have taken ages before. The price of entry used to be the restricting factor, but now we can access these technologies on the browser. It has completely democratized the process, which will change the game and open accessibility to everyone. However, there will always be a market for hand-made content. It is still obvious when humans make something versus a machine. When we automate everything, it does run the risk of all looking the same or similar. Hand-made content will still be considered ‘magical’ and ‘special’ because of its uniqueness and visceral qualities. It will be a smaller industry, but it will always exist.

Kim Davidson, President and CEO, SideFX Software

Emerging and evolving technologies, such as AI/ML or AR/VR, will not replace current solutions. Rather, they will continue to complement and improve current approaches. In digital modeling, for example, VR and ML are nice complements to procedural and interactive modeling techniques. By combining these technologies, modeling in the future will be more versatile, interactive and intuitive. At SideFX, we look to incorporate ML into future releases of Houdini as a complement to procedural workflows in modeling, environment generation, layout, animation, character effects and lighting.

Virtual production is rapidly changing current production pipelines. The benefits of virtual production are driving the growth, with the number of LED volumes likely to double in the next three to four years. While the need for post-production will shrink, digital pre-production work will increase in order to optimize the on-set shoot. Novel uses for LED volumes beyond virtual production will also help drive the growth of LED volumes. Who needs the “Metaverse” when you can spend time with friends inside a giant LED half-dome? The shortage of talent, particularly technical, is the biggest issue our industry is currently facing, and it is unlikely to subside over the next three to five years.

“Bringing innovation to the table is bringing a renewed energy to all of our work, and I don’t think there’s a more exciting time to be working in VFX than right now.”
—Johnny Fisk, President, FuseFX

A toy company robotics engineer builds a life-like, almost-living doll in M3GAN. Concept design and specialty props are provided by Wētā Workshop. (Image courtesy of Universal Pictures)

The excesses of early Hollywood are on full display in Babylon. ILM handles VFX and animation. (Image courtesy of Paramount Pictures)

Danny Turner, Executive Producer, Yannix Co., Ltd.

The industry has recovered in a big way and, in turn, Yannix enjoyed unprecedented demand for our services in 2021-22, particularly our Character Rotomation (RotoAnim) service for which we’ve seen exponential increases in client demand. There are no signs of things slowing down any time soon.

Throughout the global health crisis, Yannix remained open for business without any interruption by implementing a “seven-days-a-week” work strategy. By splitting our teams into two separate shifts, Yannix complied with social-distancing guidelines and, most importantly, we kept our people safe and healthy. Through it all, we never had a need to implement a “work from home” strategy. As the industry and consumers adjusted to the “new normal,” we focused on keeping the lines of communication open with our clients and strived to remain ready.

Mathieu Raynault, Founder & CEO, Raynault VFX

With the need for VFX productions at an all-time high, the landscape of the cinema and television industries is changing at an unprecedented pace. VFX companies have to reinvent and streamline their pipelines and technologies to account for both remote work and labor shortages. All this movement creates engaging challenges, thrilling opportunities and, without a doubt, uncertainty. We believe that keeping the human aspect of our business at the center of the VFX conversation is key to surfing this wave in the future. Raynault’s bet is to create a model where artists have greater ownership over their work, shots and assets. Our team thrives on overcoming their most complex tasks while maintaining a healthy work environment and VFX/life balance. This concept may sound cliché, but it’s actually at the core of our philosophy now and for the many years to come.

“The pressure to deliver large amounts of high-quality shots around tight deadlines [for high-end episodic productions] has set recruitment teams on fire – with no signs of slowing down anytime soon.”
—Gaurav Gupta, Managing CEO, FutureWorks Media Limited

Supernatural chiller The Pope’s Exorcist is the portrayal of a real-life Vatican exorcist. (Image courtesy of Sony/Screen Gems)

Vecna returns to terrify in the fifth and final season of Stranger Things. Vecna’s CG-animated hand was crafted by DNEG. VFX vendors include Alchemy 24, AB VFX Studios, BOT VFX, DNEG, Lola Visual Effects, Rodeo FX and Spin VFX. (Image courtesy of DNEG and Netflix)

Steve Read, Head of Studio and Executive Producer, Versatile Media Company Ltd.

Creative drives technology, and it all begins with a great story. The purpose of virtual production is to align both under one clear vision: to see results in real-time, in-camera and at the hands of key creatives.

Lensing shots on LED volumes allows directors and DPs to obtain immediate results and keep full control of both the practical and digital elements. This level of collaboration naturally brings the principal photography and VFX components together under the control of one creative drive. It promotes a transparent workflow in real-time to capture the best results possible. Our industry has evolved. Audiences today have a tremendous appetite for more quality content. The distribution and platforms are also quickly evolving.

Patrick Davenport, President, Ghost VFX

Talent remains our key priority. There’s always a shortage of artists, but this is exacerbated in the current climate. So, the focus has to be on employee retention, not just recruiting, but paying fairly (including overtime) and providing a supportive work environment and culture. The industry is still in a dizzying state of flux, with so many companies for sale, being acquired or merged. We would like to get on with the work and enjoy a stable environment, especially with everything going on in the world.

Nearly three years since the start of the pandemic, it feels like the shift to full-time WFH (Work from Home) or hybrid has become permanently embedded in our industry, which means studios have to optimize the employees’ remote work experience through technology and enhanced people support to maintain creative collaboration and productivity.

Mariana Acuña Acosta, Senior Vice President, Global Virtual Production, Technicolor

Traditional VFX pipelines will continue to evolve, moving to the Cloud, enabled by machine learning for automated data wrangling. Automation will continue to be a key piece of the VFX puzzle as it relates to performance transfers, keying, rotoscoping, de-noising, data clean-up, etc. Due to the increased demand for animated series, traditional animation will have to investigate leaner techniques, such as bypassing the 2D process altogether and jumping straight into 3D, using game engine technology to be able to scale creative output.

Given the challenges of on-set virtual production, standardization is key, which is why there’s a movement towards SMPTE 2110 (this is the equivalent of when physical tapes moved to digital files for content storage). Virtual production will continue to grow in other areas than just film and episodic. VP will see wider adoption in animation and advertising.

Hitesh Shah, Founder and CEO, BOT VFX

VFX embraces the gig economy – more seriously this time. Three years ago, most facilities (with rare exceptions) could not conceive of artists doing their work from remote locations because of the technical constraints of the infrastructure itself, let alone other factors. Then the pandemic forced a reluctant embrace of PCoIP (PC over IP) technology out of sheer desperate necessity. Soon, this tool of necessity became the transformative tool for facilities to free themselves of geographic constraints in hiring artist talent.

Pipelines and infrastructure built in one city could now leverage artist talent in far-off places without much incremental cost, setup time or process changes. Concurrent with this new enabling technology were two other factors: the surge in VFX service demand that exceeded the readily available industry capacity, and the wider social movement normalizing work-from-home. Facilities have embraced these changes by changing their operating model. They have begun recruiting remote talent to augment their base office teams.

Christophe Casenave, Head of Category Management and Sales Cinema Products, Carl Zeiss AG

Film productions in general, and VFX productions in particular, are striving for efficiency, driven by the high demand for streaming content. The well-established VFX workflows are being updated with new technologies like virtual production, which allow for the final production of pixels on set and offer new levels of flexibility. One of the major challenges these teams are encountering is the matching of the look of the CGI with the look of the physical lens, especially when the glass shows very strong characteristics, as seen with popular vintage lenses. Matching the look is mainly achieved with a lot of manual work in post-production and relies on guesswork to reproduce lens characteristics, which makes it difficult to scale and to use with real-time tools. Optimizing virtual production processes and making the images produced even more cinematic will be the major challenge in the near future.

Markus Manninen, Managing Director, Goodbye Kansas Studios

At Goodbye Kansas Studios, we see a continued high demand from clients to get visual effects work done during 2023. In particular, the episodic segment is continuing the trend of setting higher expectations of visual complexity and quality, with the effect being that clients are reaching out earlier to secure resources that are able to accomplish complex shots and scenes. We expect to see more strategic relationships between vendors and clients as a result.

The open source of core tools and capabilities will become a much more integral part of next-generation tools, workflows and processes in 2023. Virtual production is clearly here to stay, even as on-location work continues to grow during 2023.

David Patton, CEO, Jellyfish Pictures

As we navigate the expanding VFX landscape, 2023 will see us continue to build new global collaborative workflows – not only to break down the geographical barriers when it comes to sourcing talent, but also to allow us to build more speed, scale, agility and sustainability in delivering new projects.

With this in mind, Jellyfish Pictures has been harnessing the power of Cloud technology, working closely with providers such as Microsoft Azure, Hammerspace and HP to optimize our internal pipelines and boost productivity. Using modern Cloud-based solutions empower our artists to create the same standard of work as they would within a studio environment, no matter where they are in the world. In addition, the rapid rise of virtual production has also taken the industry by storm.

Gaurav Gupta, CEO, FutureWorks Media Limited

This year, we’ll be back to ‘business as usual,’ though the world we live in is not the same. The deep transformation our lives and our industry have been through will continue to permeate throughout 2023.

Thanks to the new ways in which consumers access content, there’s never been so much demand for our services. I’m talking not only about traditional feature films, but also high-end episodic productions. The pressure to deliver large amounts of high-quality shots around tight deadlines has set recruitment teams on fire – with no signs of slowing down anytime soon.