Instant Feedback: Expozure Expands its Virtual Toolset

By IAN FAILES

In late 2018, virtual production company Digital Monarch Media (DMM) was acquired by game engine maker Unity Technologies. It was a suitable fit, since DMM had been utilizing Unity in its virtual production offerings, while Unity had also been continuing to develop a suite of more accessible filmmaker tools.

DMM’s predominant virtual production solution, a part of their Virtual Film Tools suite, is a virtual cinematography simulation along with a virtual camera or V-cam system called Expozure VFT. The idea is to provide filmmakers with a window into their scene before anything is shot for real (although it can be used to re-imagine shots during and after filming, too). The company’s V-cam is a tablet that uses game controllers or the touch screen to adjust framing, lenses and other parameters.

In recent times, DMM’s tools have been used for films such as Stowaway, Greyhound, Blade Runner 2049, Ready Player One and The Jungle Book. Depending on the film, the virtual camera is part of previsualizing shots in pre-production, or working out shots on a motion-capture volume or greenscreen set, or generating final camera moves in post.

At SIGGRAPH 2019, Digital Monarch Media (DMM) co-founder Habib Zargarpour (center) worked with DOP David Stump, A.S.C. to craft a real-time rendered scene in Unity in front of an audience using the Expozure virtual production toolset.

“We’ve been pushing on two fronts. One is full-on use of the tool on a mocap stage in a high-end situation. At the same time, what we have also been pushing for is having everything be easy to use by non-technical people. We want the DPs and the directors to just grab this stuff and run with it, put scenes together and start shooting. Simplicity is important.”

—Habib Zargarpour, Co-founder, DMM

A still from an astronaut Expozure demo ‘filmed’ in real-time.

The astronaut scene made use of motion-captured characters, with framing, lighting and editing able to happen on the fly.

This screen capture from the Expozure V-cam shows how lighting can be adjusted in real-time.

A virtual camera rig rail can be established in Expozure.

“One person was able put [the astronaut demo] together in five days. We bought two models of the characters, had them rigged, got the environment from the Unity Asset Store, learned HDRP, learned Visual Effect Graph, and learned all the volumetric things you could now get with lighting. We did the mocap for all three characters using an inexpensive inertial suit in about 15 minutes. Then we had cinematographer David Stump, ASC come and shoot this scene live. It took him 20 or 25 minutes to shoot the whole thing. Plus we had live music mixing.”

—Habib Zargarpour, Co-founder, DMM

THE STATE OF PLAY WITH DMM’S TOOLS

By letting the filmmakers ‘grab hold’ of their own virtual camera, and by pushing everything through the real-time Unity engine, DMM’s mandate is instant feedback and providing a means for many shot iterations. What has changed since the Unity acquisition has been access to improved workflows in the game engine, including realistic VFX, volumetrics and real-time ray tracing.

“We’ve been pushing on two fronts,” says Habib Zargarpour, who co-founded DMM with Wes Potter, and now operates out of locations in Los Angeles and Vancouver. “One is full-on use of the tool on a mocap stage in a high-end situation. At the same time, what we have also been pushing for is having everything be easy to use by non-technical people. We want the DPs and the directors to just grab this stuff and run with it, put scenes together and start shooting. Simplicity is important. The ease of use is important, as is the standalone functionality, which is something unique to our tools, meaning that everything runs in run-time.”

The V-cam is now more streamlined than before, too. It’s packaged with a 3D printed holder that allows for two game controllers and the screen to be hand-held. The setup enables the ingestion of a CG scene, or as the use of a simul-cam to do real-time compositing on set or location. A user can establish camera framing and moves and be able to move characters or props around the scene.

A demo DMM conducted at SIGGRAPH 2019 involved CG astronauts and was intended to show how quickly a virtual production setup could be orchestrated from scratch. The astronauts, imagined in some kind of confrontation, were mocap’d inside a space station. The demo took advantage of the current DMM virtual production tools and Unity’s newer photorealistic rendering capabilities, including the High Definition Render Pipeline (HDRP) and the Visual Effect Graph.

“One person was able put this together in five days,” relays Zargarpour. “We bought two models of the characters, had them rigged, got the environment from the Unity Asset Store, learned HDRP, learned Visual Effect Graph, and learned all the volumetric things you could now get with lighting. We did the mocap for all three characters using an inexpensive inertial suit in about 15 minutes. Then we had cinematographer David Stump, ASC come and shoot this scene live. It took him 20 or 25 minutes to shoot the whole thing. Plus we had live music mixing.”

During the demo, the V-camera was used to frame the scene while other devices, including a Pixel Phone, were used to puppeteer the astronauts’ spaceship landing and even maneuver a light source all while Expozure recorded the action for editorial, or even live sequencing.

SHIPS AT SEA, AND OTHER VIRTUAL FILMMAKING

For the Aaron Schneider film Greyhound, DMM’s Expozure was used to visualize the at-sea scenes. Real-time rendered photorealistic ocean waves aided in giving the filmmakers an extensive preview of what their final shots – later completed in VFX – would look like. Shots could be imagined as hand-helds, or cameras anchored to virtual camera ships to simulate the realistic camera reactions to the motion at sea – aerial drone fly-bys and helicopter shots were not realistic enough.

And since the previs was done for that film, DMM’s tools have been updated to enable greater photorealism, such as more volumetric lights. Zargarpour suggests that this extra detail only adds to the director, DP, or VFX supervisor’s experience in imagining the scene.

“They are able to make decisions about what they’re going to shoot in a lot more detail, especially if we’re able to bring realistic lighting into the world,” he says. “The goal with getting the rendering to be super high-fidelity is now to do final pixel. We’re looking to be able to actually put this on an LED screen and have somebody in the foreground as though they’re on a ship and have the background be virtual. Then as you walk around, the parallax gets translated to the screen.”

That’s something that the makers of Disney+’s The Mandalorian have been doing, of course, with LED screens in a dedicated volume. The approach now seems to be destined to be a clear part of the future of filmmaking.

The upcoming Stowaway from Director Joe Penna relied on Expozure’s V-cam for generating previs of in-space sequences. “We planned to shoot the previs over two weeks, but they used it so quickly that it was done in a week,” says Zargarpour, noting a similar fast turnaround of previs shots had been achieved on Greyhound compared to the traditional approach of animating shots in Maya or other similar content creation tools.

As Zargarpour noted above, DMM’s Expozure is not just for high-end productions. The team recently worked with some independent filmmakers on crafting a digital set of a church interior that matched a real shooting location. This way the filmmakers could scout the set and plan shots which would involve several visual effects.

“We scanned the sets via photogrammetry and brought them into the system,” outlines Zargarpour. “The director came and they ‘shot’ 75 previs shots from that. It meant that when it came to the actual shoot, they were going to be much more informed, even if they hadn’t done previs, because they’d been able to do a one-to-one set scout.”

DMM co-founders Wes Potter (left) and Habib Zargarpour.

Inside the offices of DMM.

“They are able to make decisions about what they’re going to shoot in a lot more detail, especially if we’re able to bring realistic lighting into the world,” he says. “The goal with getting the rendering to be super high-fidelity is now to do final pixel. We’re looking to be able to actually put this on an LED screen and have somebody in the foreground as though they’re on a ship and have the background be virtual. Then as you walk around, the parallax gets translated to the screen.”

—Habib Zargarpour, Co-founder, DMM

Habib Zargarpour reviews a previs’d scene on a tablet.

The Expozure V-cam is made up of a rig with two game controllers.

“We scanned the sets [of a church interior] via photogrammetry and brought them into the system. The director came and they ‘shot’ 75 previs shots from that. It meant that when it came to the actual shoot, they were going to be much more informed, even if they hadn’t done previs, because they’d been able to do a one-to-one set scout.”
—Habib Zargarpour, Co-founder, DMM

VIRTUAL PRODUCTION: THINGS TO CONSIDER

One of the key aspects of DMM’s Expozure is that it is aiming to replicate what is already done on a real film set with real, practical filmmaking equipment. That way anything imagined in previs or ‘synthetic’ form can be repeated with this real equipment, or, if done as a CG shot, does not feel out of place with the rest of any live-action photography.

“I’ve always been a huge proponent of performing a camera move as opposed to keyframing,” states Zargarpour. “I like people to actually hand-hold the virtual camera and move it. We also have a shoulder-cam where you put it on your shoulder. That’s something that was used on Greyhound where the camera follows a ship. The director wanted to follow the movement as though he was on another ship, following the bow and have that weight.

“We’re also working to bring in any number of tools that directors like to use,” adds Zargarpour. “Things that have an analog equivalent. All of our cameras use physics internally. So when you’re in dolly mode, you’re pushing the dolly and you actually get the sensation of the inertia. Also when you do use a hand-held camera with the V-cam, you still have the option of smoothing it out to give it a Steadicam feeling.

“We’ve been deploying the tools at various studios and working with people to understand what it takes for something to work without us being there. So any time we have a new feature we’re always thinking, ‘Okay, is this going to over-complicate things? What’s the easiest way to do it?’ We feel strongly that we have to keep simplifying things and keep making them accessible.”