VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


April 15
2024

ISSUE

Spring 2024

How to Shoot and Edit Animation Using Live-Action Virtual Production

By ADAM MAIER and PAUL FLESCHNER, Reel FX, and CONNOR MURPHY, The Third Floor

Edited for this publication by Jeffrey A. Okun, VES Abstracted from The VES Handbook of Virtual Production Edited by Susan Zwerman, VES and Jeffrey A. Okun, VES

Introduction: The Creative Story Process

At its heart, this is a conversation about the creative story process itself. The use of real-time tools in animation enables the convergence – or intersection – of animation and live-action techniques. Many trailblazing films have led the way, from Beowulf (2007) to The Adventures of Tintin (2011), all before the advent of real-time technology as it exists now. Over the last decade, there have been significant advances in game engines’ readiness for real-time (live) visualization on set and in camera and, in some cases, “final pixel” applications. Stylized hand-key animation can also be produced with this approach.

As real-time tools improve, their artistic and workflow benefits become impossible to ignore. The development of Virtual Production for animation, or “real-time animation,” can be viewed as a new and complementary toolset for the animation professional. The framework of understanding also needs to integrate the impact of cinema’s live-action heritage suddenly made available to the animation professional. This live-action mindset will not be right for all animated projects, and great content will continue to be made in the traditional way. Whatever the approach, the benefits of real-time are compelling enough for a deep investigation. Ultimately, the goal of animation is telling a good story with nice pictures. Real-time animation offers a new road to that end, with some surprising benefits.

Figure 4.2 Super Giant Robot Brothers! (2022). (Image courtesy of Reel FX Animation)

Figure 4.2 Super Giant Robot Brothers! (2022). (Image courtesy of Reel FX Animation)

As the name implies, real-time animation accelerates the production process, which can lead to creative and/or cost s avings. This acceleration does not equally impact every phase of production, but it does affect most and ripples through the entire production. There are three primary means from which this acceleration occurs:

  • Immediacy of Visualization
  • Increased Creative Iterations
  • Making Creative Decisions in Context

The first (and the catalyst for the rest) is the immediacy of visualizing and/or rendering using a real-time renderer that renders frames in milliseconds, allowing artists to work without waiting, saving time and money. Final renders for an entire 22-minute episode of Super Giant Robot Brothers! (Reel FX/Netflix 2022) finished overnight on five workstations, as opposed to the traditional 500-node render farm. It is worth noting, however, that final frames can still be rendered with a traditional renderer. While real-time renderers cannot yet achieve the quality of an offline renderer, they inch closer every day. Visualizing in real-time and translating those decisions to an offline renderer can become quite complex, and speed will be lost in later departments as they lose these real-time render speeds. Take all factors into consideration when choosing the final renderer.

The second acceleration is the reduction of the time between creative iterations. The underlying assumption is that more iteration will lead to a better story/product. It can be said that this rapid visual ideation provides a creative team with “more bites at the apple.”

The third acceleration is improved context when making collaborative creative decisions. Teams can collaborate live and parallelize tasks with a real-time workflow. This saves time. Because renders are instantaneous, live “real-time” reviews are possible. Creative work that was previously accomplished asynchronously can now be realized simultaneously, and artists can react to one another. For example, camera work can interplay with performance during motion capture, or lighting and camera can interplay during the virtual camera phase. Decisions are made with more context up front, leading to fewer changes and surprises downstream.

Members receive a 20% discount with code: HVP23 Order yours today! https://bit.ly/VES_VPHandbook



Share this post with

Most Popular Stories

THE CONSEQUENCES OF FALLOUT
15 April 2024
VES Handbook, Virtual Production
THE CONSEQUENCES OF FALLOUT
Westworld team brings Megaton Power to game adaption.
SINGING PRAISES FOR UNSUNG HEROES
15 April 2024
VES Handbook, Virtual Production
SINGING PRAISES FOR UNSUNG HEROES
Recognizing ‘hidden’ talent pivotal to making final shots a reality.
AGING PHILADELPHIA FOR THE WALKING DEAD: THE ONES WHO LIVE
03 April 2024
VES Handbook, Virtual Production
AGING PHILADELPHIA FOR THE WALKING DEAD: THE ONES WHO LIVE
The final season of The Walking Dead: The Ones Who Live follows Rick Grimes and Michonne Hawthorne.
LAS VEGAS’ SPHERE: WORLD’S LARGEST HIGH-RES LED SCREEN FOR LIVE ACTION AND VFX
15 April 2024
VES Handbook, Virtual Production
LAS VEGAS’ SPHERE: WORLD’S LARGEST HIGH-RES LED SCREEN FOR LIVE ACTION AND VFX
World’s largest high-resolution LED screen immerses audiences.
NAVIGATING LONDON UNDERWATER FOR THE END WE START FROM
05 March 2024
VES Handbook, Virtual Production
NAVIGATING LONDON UNDERWATER FOR THE END WE START FROM
Mahalia Belo’s remarkable feature directorial debut The End We Start From follows a woman (Jodie Comer) and her newborn child as she embarks on a treacherous journey to find safe refuge after a devastating flood.