HOW WRITING FOR VIRTUAL PRODUCTION HAS FLIPPED THE SCRIPT

By OLIVER WEBB

“The sky is the limit [with virtual production], to the kinds of worlds we can create for any given story, to the places on this planet or this galaxy we want to explore without ever having to set foot on a plane, or even to the worlds that only exist in our minds. Especially when writing sci-fi – which was my experience – or any genre in period; you can go back to the Ice Age if you want to.”

—Juliana Lima Dehne, Story Editor/Writer, 1899

Combining CGI, real-time visual effects and live-action filming, virtual production technology has boomed in the last few years. With The Mandalorian utilizing the technology, other productions have since followed suit, including 1899, Star Trek: Discovery, House of the Dragon and Avengers: Endgame, among others. The cutting-edge technology has not only become an alternative to on-location filming, but it has opened up endless possibilities for filmmakers, as well as offering a more immersive world for actors to surround themselves in. While there are many benefits to virtual production technology, when it comes to the creative coverage of a scene, it also has its limitations.

Michael Shelton, Creative Director at Pixomondo, oversees projects from prep through final delivery. “Depending on Pixomondo’s role on a given project, this would include on-set supervision as well,” Shelton notes. “I end up wearing many hats and sometimes find myself on the box lighting shots or doing a bit of character animation or matte-painting work, which I love. Anytime you can walk onto a set and see what the end result will look like while still in principal photography is an advantage to everyone involved. Aside from the wow factor, it gives both the VFX team and the production team the best opportunity to blend the live action into the created environments. Actors especially benefit from seeing the world built around them, in a way it removes the burden of needing to imagine what their surroundings look like.”

Virtual production technology has altered the writing process when it comes to writing for films and series that use the technology. “The difference – or at least in my singular experience working on 1899 – was that much of the rewriting in post is happening in pre-production or during production latest,” says Juliana Lima Dehne, who was Story Editor and wrote Episode 5 of the Netflix series, a show that relied on virtual production to create its vivid landscapes.

“One of the reasons for it is that locations – for example – are set. You can’t change your mind; nor will the controlled environment change once you start production. It’s all definitive,” Dehne adds. “The sets and the images projected onto the LED screens have already been designed, and the soundstage has already been dressed to match it. It’s like a game. You have to work with what’s there. You can’t decide to go off script in the middle of a game because you want to enter a door that doesn’t open because nobody designed what’s behind that door. I remember having these discussions in the writers’ room about locations, how the technology worked in terms of where to place characters in any given location we were working with. There was a rule of three, where it was like here’s the wide shot we’re working with, the ‘bigger picture,’ and here are the two points we can move towards, zoom in or enter within this wider shot. Scenes have to happen and make sense within that. So, I think you can think of it as doing post in pre-production, and the scripts have to be as informative as possible. More than just blueprints, they kind of had to be the framing of the house already, and locations have to be locked in, in that sense. In case you’re wondering, pre-production is extensive and intense. As a staff writer, that was the biggest difference. It was like, don’t start pitching a scene in a burning skyscraper when they’ve already designed the abandoned cottage in the woods.

Even if for maybe the character in that moment, a burning skyscraper would’ve conveyed the character’s inner struggle/emotional state/turning point much better, it’s literally too late. Granted there was still a window to change locations during the writers’ room period, because the designing of the world was happening simultaneously, but once they were done and we were still writing, it was over. It definitely forces you to be more diligent with those early drafts.”

For Emma Needell, writer and director of the short film “Life Rendered,” virtual production greatly affected the writing process. “This was because I embraced the tools of virtual production for story development specifically. I first learned how to use Unreal Engine – which is free to download and use – then I created an animatic of my short film in the program, experimenting with pacing, cinematography and music all before we began pre-production in earnest,” Needell remarks. “As a filmmaker, the ability to iterate through ‘drafts’ of your film via an animatic is extremely advantageous. The process gave me the ability to visualize and communicate the intent of each scene with key members of my production team, because I workshopped it on my own, like a draft of a script. The flip side, of course, is ensuring that you don’t get too attached to the animatic. You must allow for happy accidents and your team’s expertise to help shape the final film.”

Despite the many benefits of virtual production technology, the technology can ultimately be time-consuming. “In a live-action production, what you shoot is what you get, and the film is then fully crafted in editing. With virtual production, there are extra steps in both pre and post to finesse the image in Unreal Engine, which can take a long time, especially if you’re working on an indie project like I was,” Needell explains. “It’s pivotal to work with a producer who understands the nuances of virtual production, because it’s very different from traditional filmmaking.”

According to Dehne, one of the advantages of virtual production, when it comes to creative coverage of a scene, is in the types of stories you can tell that wouldn’t be possible if you had to shoot on location, or would be too costly to do so. “The sky is the limit, to the kinds of worlds we can create for any given story, to the places on this planet or this galaxy we want to explore without ever having to set foot on a plane, or even to the worlds that only exist in our minds,” Dehne details. “Especially when writing sci-fi – which was my experience – or any genre in period; you can go back to the Ice Age if you want to.”

Dehne also notes the impact virtual production has on actors. “Unlike with a greenscreen, it helps them focus on the emotion and intensity of scenes, because nobody has to waste brain cells trying to imagine the set while saying lines. The set, the landscape, the world, it’s already there in front of your eyes,” Dehne observes.

“I embraced the tools of virtual production for story development specifically. I first learned how to use Unreal Engine… then I created an animatic of my short film in the program, experimenting with pacing, cinematography and music all before we began pre-production in earnest.”

—Emma Needell, Writer/Director, “Life Rendered”

“On set, you’re watching monitors of a scene being shot on a ship in the middle of the ocean, some people even got seasick if I remember correctly, but you’re in a studio somewhere in Berlin. It’s quite impressive. What would take extensive time to marry in post-production – the real and the digital, VFX – is literally happening in real-time as you shoot. You can basically adjust as you go. Plus, for all your exterior shots you can control lighting, the weather, unlike shooting on location somewhere. In Germany, for example, I remember shooting this short film when I first moved over from New York City, and we shot it in April. We literally had all four seasons in one day, and then we were stuck trying to make it work in post and overspending to reshoot because snow, rain, sun and clouds in an important character establishing two-minute scene is an absolute nightmare. Having complete control of the environment is definitely one less problem to worry about as a creative.”

When it comes to limitations of the creative coverage of a scene, Needell argues that there are trade-offs to every tool and technology in film production. “The limitations to getting coverage in virtual production can be offset by the ability to be on a climate-controlled stage where you control every aspect of the set. Magic hour lighting from the first shot to the martini shot? No problem. You also don’t have to worry about location changes, since you can just load up your next scene onto the stage with a few clicks of a button. However, bugs in the technology can slow down your days and cut into the time you need to maximize coverage. For instance, during our production, the AC in the building went out one day, and the computer servers got so overheated we had to shut down production for an afternoon. But that’s film production! It never goes according to plan, and, as a filmmaker, you need to be ready to adjust and pivot given what the conditions of the day are,” Needell adds.

Dehne stresses the importance of being well-prepared as both knowledge and experience of the technology are vital. “In terms of angles and shots, lighting and mood, within a defined space it can be actually creatively freeing since you’re seeing the result in real-time and can adjust accordingly,” Dehne explains. “Beyond that defined space, it’s challenging. You can’t suddenly decide to peek in through a window projected on your LED screen if nobody designed the reverse of that.”

“Ironically, in this case, virtual production was the only possibility to make it happen. I believe they made the show they wanted because of it. I do remember the excitement in the writers’ room when writers would pitch crazy ideas like: Could there be a graveyard of ships from previous failed simulations? Is there a world where there isn’t just one trapdoor but a million of them all over the ship? Or how many character ‘memories’ could interact with each other in a single episode. The technology itself was a beast everyone had to tame. It’s complex, requires time to understand, to learn, but what it brought and helped the creators achieve was well worth the initial trepidation,” Dehne concludes.