VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


June 06
2024

ISSUE

Spring 2024

SETTING THE STAGE FOR LED VOLUMES

By TREVOR HOGG

Autodesk’s Paolo Tamburrino believes that there’s room for software innovation when it comes to making LED volume shoots more accessible. (Image courtesy of Autodesk

Autodesk’s Paolo Tamburrino believes that there’s room for software innovation when it comes to making LED volume shoots more accessible. (Image courtesy of Autodesk)

As with any tool in the film and television industry, the LED volume is being shaped by technological advancements, budgetary requirements and artistic demands, which means the application and capability of the virtual production methodology is constantly changing.

“From the outside looking in, it seems that the standardization of the previs process with LED volume production will be integral to broadening adoption of LED volumes,” states Tim Walker, Sr. Product Manager at AJA Video Systems. “Most large LED volume studios develop and leverage at least some of their own tools to fit their needs because there isn’t one cookie-cutter solution. However, once you hit ‘cookie-cutter status’ much wider adoption is possible as the costs and expenses come down.”

Some on-set roles have become even more critical within LED volume such as the DIT. “The DIT is responsible for working with the DP and/or director to ensure what is being captured will meet the demands of final color by monitoring the desired creative look on set for dailies, post-production, visual effects and ultimately final color grade,” Walker notes. “The volume introduces another variable in creating and maintaining that look, which the DIT will ultimately have some responsibility for and has the ability to assist in making any on-set color grading decisions that will help the overall production succeed early in the production process.”

StageCraft was utilized by NBCUniversal to promote the 2024 Olympic Games in Paris.(Image courtesy of ILM)

StageCraft was utilized by NBCUniversal to promote the 2024 Olympic Games in Paris. (Image courtesy of ILM)

“We have a teaming agreement where we can use the stage and [Dark Slope studio] can use Houdini, We want to push Houdini to be more real-time through this partnership, and we’re going to learn about production that we’re only touching with our technologies.”

—Kim Davidson, President/CEO, SideFX, and VES Board Chair

When it comes to whether the software is keeping pace with the hardware development, Paolo Tamburrino, Senior Manager, Strategic Business Development at Autodesk, remarks, “Yes and no. Heavy investments have been made by Epic Games in Unreal Engine. Unreal Engine has been used heavily primarily for in-camera visual effects [ICVFX], but there is room for more innovation across the software community to make LED volume shoots more accessible.” Autodesk works closely with Epic Games. Says Tamburrino, “Our main collaboration is with Epic’s Unreal Engine and Maya Live Link, which allows artists to create assets in Maya and visualize them in real-time in Unreal Engine.”

Brompton Technology offers features like ShutterSync and Frame Remapping, which expand upon what filmmakers can do with virtual production. (Image courtesy of Brompton Technology)

Brompton Technology offers features like ShutterSync and Frame Remapping, which expand upon what filmmakers can do with virtual production. (Image courtesy of Brompton Technology)

The emergence of Virtual Art Department services has reshaped the visual effects process and the types of stories being told. Explains Tamburrino, “Firstly, it can help screenwriters understand the worlds they’re writing for, how actors will interact with them and enhance the creative ideation of a film. Secondly, visual effects and art departments are involved at much earlier stages of pre-production, basically when the concept and the ideation starts, and that helps filmmakers have visual reference to tell better stories. In cases where content is captured on greenscreen, it’s very difficult even for established screenwriters, directors and DPs, not to mention all the other departments working traditionally in the film set, to imagine what the world might look like. Finally, shooting on an LED volume can give creative control back to the filmmakers, which has been lost to some degree with the traditional process in the 12-plus months of post-production/VFX after principal photography, where a film’s creative process often ends up out of the creative control of the filmmakers.”

Virtual production excels at driving shots, as demonstrated in this partnership between BromptonTechnology and Mushang Studios. (Image courtesy of Mushang Studios)

Virtual production excels at driving shots, as demonstrated in this partnership between Brompton Technology and Mushang Studios. (Image courtesy of Mushang Studios)

Advertising is an enormous growth area for virtual production.(Image courtesy of 80six)

Advertising is an enormous growth area for virtual production. (Image courtesy of 80six)

An example of accelerated volumetric video by Arcturus.(Image courtesy of Arcturus)

An example of accelerated volumetric video by Arcturus. (Image courtesy of Arcturus)

“The big thing that has changed is how easy it is to do this at all,” observes Adrian Jeakins, Director of Engineering at Brompton Technology. “Back on The Mandalorian Season 1, it was the industry’s brightest minds figuring out how to get all of this to work, and there were no YouTube tutorials to follow. Now, the barrier to entry is much lower, which is amazing. For Brompton, as the LED part of the process, we’ve seen a rapid evolution in terms of the amount of flexibility and creative control that you had on those early volumes, to now. Initially, there was just a narrow range of settings that would work, and anything beyond those would break. Furthermore, you could only have one camera. We now have features like ShutterSync and Frame Remapping, which enable filmmakers to do so much more with virtual production.”

LED volumes excel in capturing certain shooting scenarios. Jeakins adds, “Virtual production is indisputably the best method for driving shots that are challenging on greenscreen because you have reflections everywhere. Again and again, we’ve seen filmmakers try virtual production for driving shots, and they love the experience. Generally, virtual production makes a lot of sense for episodic television because you’re pushing for feature film quality but don’t have a feature film budget. And you can easily reuse the same environments throughout a season or more. I’d say one of the biggest areas for growth is in traditional broadcast studios. Virtual production greatly improves the experience for talent. Advertising is also an enormous growth area. Advertising is a fast-paced business, and the creatives have different requirements in terms of photorealism, so they can take advantage of the flexibility of this technology.”

Among the technological advances is the ability to populate LED walls with volumetric video characters and crowds. “It’s exciting to see people realize what LED walls are capable of on an ongoing basis,” states Piotr Uzarowicz, Head of Partnerships and Marketing at Arcturus. “LED walls were limited in that you could put people on them because the only way to do so was as two-dimensional characters. If the camera moved, it destroyed the illusion of a person. But now that the characters on that wall can be volumetric and three-dimensional, it allows for the camera to shift and still represent that person on the LED wall in a truthful way. If you put them as secondary or background or distant characters, you won’t be able to tell the difference between a volumetric character and a real person.” To assist with the rendering, not everything is treated with the same level of detail. Uzarowicz says, “The software automatically renders characters that are closer to the camera at a higher resolution than the ones which are further away. Depending on how the camera moves, the characters are reanimated to higher or lower resolutions.” Scenes can be treated more realistically. Uzarowicz adds, “For example, where it might have been complicated to reproduce a city street with a café in the background and our characters are on the corner, that can now be reproduced realistically with a LED wall because we can populate that scene as if it were Manhattan. Lots of people running around back and forth, and we don’t need to use extras, which makes it more affordable for productions coming into that space, and more manageable in that you don’t need as large a crew to deal with hundreds of extras. You can manage all of those things in pre-production and have that realism for your principal actors during production so they can see the environment around them and have something to respond to.”

“AI-generated content will be huge in this area,” believes Ella Kennea, Production Manager at Dimension Studio. “We’re already seeing huge strides in upscaling technology, but we’re also excited about the progress in generative AI in quickly building assets and responding to creative feedback.” All departments need to become familiar with the technology. “It can’t be stressed enough that education and more hands-on experience for production heads of departments is crucial to the success of virtual production,” remarks James Franklin, Virtual Production Supervisor at Dimension Studio. “Also, education specifically around the content creation process. Directors and DPs have an enormous amount of creative opportunity there; they should definitely be getting involved earlier!” There are those educating themselves. “We’re seeing DPs, DITs and directors using bespoke tools on set, which enables them to directly make the changes they want,” Franklin notes. “Developments to Unreal Engine allow us to use high poly-count assets which is great for quick LiDAR scans. And as the industry matures, procedures are becoming more established. There is now much more experience on set and less of the unknown.” The spotlight should be solely on LED volumes.

The DNEG Virtual Production team creates a crowd scene for a gladiator battle. (Image courtesy of DNEG)

The DNEG Virtual Production team creates a crowd scene for a gladiator battle. (Image courtesy of DNEG)

Dark Slope and SideFX have collaborated to launch an advanced virtual production stage, the second largest LED volume in Toronto. Dark Slope had their launch party in Toronto in early June. Speaking at the event was Kim Davidson,President and CEO of SideFX and current VES Board Chair. (Image courtesy of SideFX)

A test pattern in a Magicbox Superstudio. (Image courtesy of Magicbox)

A test pattern in a Magicbox Superstudio. (Image courtesy of Magicbox)

“Virtual production is an integrated production pipeline that starts in pre-production and carries through to post,” Franklin states. “We can previsualize scenes, more effectively plan shoots and provide savings with the data we capture. Everything that we create digitally is useful throughout the pipeline. So far, we’ve only scratched the surface. There are many more opportunities waiting to be discovered.”

Virtual production is an integrated production pipeline that starts in pre-production and carries through to post-production. (Image courtesy of DNEG)

Virtual production is an integrated production pipeline that starts in pre-production and carries through to post-production. (Image courtesy of DNEG)

Autodesk assisted with the LED stage shoot for Season 1 ofStar Trek: Strange New Worlds. (Image courtesy of Autodesk)

Autodesk assisted with the LED stage shoot for Season 1 of Star Trek: Strange New Worlds. (Image courtesy of Autodesk)

With Dimension Studios, Arcturus makes use of an open system that accommodates a variety of cameras, hardware and servers.(Image courtesy of Arcturus)

With Dimension Studios, Arcturus makes use of an open system that accommodates a variety of cameras, hardware and servers. (Image courtesy of Arcturus)

Collaborating to launch an advanced virtual production smart stage in Toronto that will consist of a 75-inch volume and an AI-enabled pipeline is virtual production-focused studio Dark Slope and SideFX. “We have a teaming agreement where we can use the stage and they can use Houdini. We want to push Houdini to be more real-time through this partnership, and we’re going to learn about production that we’re only touching with our technologies,” remarks Kim Davidson, President and CEO of SideFX and current VES Board Chair. “Epic Games is an investor in SideFX, and we have a great relationship with them. But it’s another thing to have a company in Toronto to go to and see how Houdini performs in real-time or in conjunction with Unreal Engine because they’re Unreal Engine experts there. Virtual production is not going to be a bit part of the entertainment industry, but we’re always trending on the cutting-edge stuff, so for us it will be a case study for how virtual production plays out, and then you can usually trickle some of those technologies down to the mainstream, and mainstream still means high-end for us.”

“LED walls are only as good as the people and software that run them,” states Raja Khanna, Co-Founder and Executive Chair at Dark Slope. “2023-2024 is a tipping point as this tech has now been around for a number of years. It has been used, there are experts, new workflows and pipelines, and software like Unreal Engine has released updates that focused specifically on virtual production, which is essentially the art of using these types of walls and stages. All of that has combined to bring us to a place where it is finally at the stage where we can deliver a lower cost but producing the same quality.” The talent pool has dramatically increased. Khanna explains, “If we were talking three years ago, everybody was building a wall, but there was no one to run them. Today, almost every media college program around has a virtual production program, and they are graduating people who understand this technology, Unreal Engine, and the workflows, maybe not as well as they will in a few years; however, the version 1.0 of all those academic training programs has been released.”

What remains as a major strength of the LED volume is the ability to remove the guesswork for the cast and crew. “One of the biggest things is being able to be fully immersed in an environment as they’re telling the story and being able to be inspired by what that space looks like, rather than walking into a greenscreen environment and working with your actors to imagine things,” observes Shivani Jhaveri, Virtual Production Producer at ILM. “A lot of our filmmakers have come back with the feedback that it helps to enhance the performances of the actors and allows our DPs to be fully immersed in the process as well from the get-go in the early stages. On every show, people are finding more use cases for it, and what makes StageCraft special is that we’ve been doing this for a number of years. One of the key pieces when we’re introducing new filmmakers to the process is, just as much as we tell them how to use the technology, we are also able to show them examples of where it might not have worked as well in the past. It’s never a one-size-fits-all scenario. More often than not, we’re actually seeing filmmakers use virtual production in their pre-production phase now.”

“You want to embrace the things where you’re leaning into the efficiencies of virtual production. So, being able to scout locations and visual effects things, to be able to mock up sets quickly to look at a design plan, then start lensing it with a camera and say, ‘Oh, wait. If we don’t build that side of the building, we’re going to be shooting into the backside of it the entire time’ [is an important benefit],” remarks Chris Bannister, Executive Producer, Virtual Production at ILM. “Sometimes, you’re doing things with virtual productions that are pushing the technology to its absolute limits, and other times it’s figuring out what are the efficient tools and how do you tap into them. For all of the tools, the people are the most important part to any successful virtual production, people who understand the tools and are able to collaborate with the creatives to say, ‘Here is how we can be useful.’ The thing is being able to embrace it less as one [solution] and more about, what are the pieces that you can plug into? Similarly, what we want to embrace on LED volumes is that you can shoot lots of takes, shoot in several directions, get a lot of freedom in the edit and be able to change things later that you might not have if you shot large sequences on greenscreen. It’s not like we’re not using practical instruments, but it allows a lot of flexibility for the DP to audition ideas and work with those tools.”

Democratization is taking place when it comes to access to virtual production technology. “When I say democratization, I mean taking that technology outside of Hollywood and making it accessible anywhere,” states Brian T. Nowac, Founder and CEO of Magicbox. “The price point is in many cases 1/10 or 1/100 of the price to build or rent these large facilities, making it accessible to almost any level of production. Our starting price point is $10,000 a day. It costs that much to rent a restaurant to do a shoot in. We’re looking to change the usability from extremely complex that it began as, into a more user-friendly and recognizable toolset that almost anyone who is working in this industry can come in, sit down and start using our virtual production solution almost instantly. In making this technology mobile and accessible at a price point, we’re opening up new markets to use this technology. We’re not specific to motion picture or video production anymore. We’re now being used as an experiential, event and educational device. In terms of lingo, that is going to vary for us based on the industry that our solution is being used by. If it’s the motion picture industry, obviously we want to know that vernacular and production on-set etiquette with the people who are operating our products. But it is also important to note that we’re not ILM. We’re not providing the LED volume and VAD [Virtual Art Department]. Our solution is more like a camera solution. We’re a piece of equipment with an operator right now, but eventually that will change. There is a certain learning curve that needs to happen. After that learning curve is achieved, you might go to a lot, pick up a Magicbox yourself and drive off with it, or call up or use the app and have one delivered to you, and you’ll know how to use it.”

A major strength of the LED volume is the ability to remove the guesswork for the cast and crew. (Image courtesy of ILM)

A major strength of the LED volume is the ability to remove the guesswork for the cast and crew. (Image courtesy of ILM)

Rather than having to go to a virtual production studio, Magicbox offers a mobile solution that can go to the clients rather than the other way around.(Image courtesy of Magicbox)

Rather than having to go to a virtual production studio, Magicbox offers a mobile solution that can go to the clients rather than the other way around. (Image courtesy of Magicbox)

Magicbox is about 1/10th the price to build or 1/100th the price to rent a virtual production studio.(Image courtesy of Magicbox)

Magicbox is about 1/10th the price to build or 1/100th the price to rent a virtual production studio. (Image courtesy of Magicbox)



Share this post with

Most Popular Stories

CHECKING INTO HAZBIN HOTEL TO CHECK OUT THE ANIMATION
16 July 2024
Virtual Production
CHECKING INTO HAZBIN HOTEL TO CHECK OUT THE ANIMATION
Animator Vivienne Medrano created her series Hazbin Hotel which has received 109 million views on her VivziePop YouTube Channel.
SPECIAL FOCUS: VFX IN EUROPE
06 June 2024
Virtual Production
SPECIAL FOCUS: VFX IN EUROPE
Despite setbacks, VFX companies look for year-end bounce.
EXPANDING THE MIND FOR INSIDE OUT 2
06 June 2024
Virtual Production
EXPANDING THE MIND FOR INSIDE OUT 2
Expanding the cast of Emotions and vocabulary of shapes.
ADDING AN EXTRA LAYER OF CREATIVE TO GODZILLA X KONG: THE NEW EMPIRE
08 May 2024
Virtual Production
ADDING AN EXTRA LAYER OF CREATIVE TO GODZILLA X KONG: THE NEW EMPIRE
Godzilla x Kong: The New Empire follows Godzilla and King Kong as they face a new threat hidden deep within the planet.
IMAGINATION REIGNS SUPREME FOR KINGDOM OF THE PLANET OF THE APES
06 June 2024
Virtual Production
IMAGINATION REIGNS SUPREME FOR KINGDOM OF THE PLANET OF THE APES
More nuanced, human-like digital cast in an upside-down world.