Beyond the Park in WESTWORLD with VFX Supervisor Jay Worth

By IAN FAILES

After two seasons of HBO’s Westworld set mostly inside the show’s complex Wild West-themed artificial park, the latest season has moved out into the ‘real’ world. We now get to glimpse a whole new futurescape and the complex people – and machines – within it.

For the visual effects team on Season 3, in particular, this meant contributing a wealth of new environments, vehicles, robots and action. It also offered up new opportunities for how certain scenes could be imagined. For instance, a number of flight sequences and office environments made use of LED walls and real-time rendering technologies to provide interactive backgrounds during filming.

The new locations of Season 3 were in many ways a continuation of the VFX design challenge faced by Visual Effects Supervisor Jay Worth. “Every inch of every frame is designed,” he says. “You can’t really point the camera anywhere in the [Westworld] world without [visual effects], the art department or costume having to touch it. Even when we were back in the park, we had to remove everything in the sky. We’d be up on a hillside and say, ‘Oh this is great!,’ and then all of a sudden you see a random telephone pole in the background. Those times happen a lot where we’re at now this season.”

To handle the large visual effects assignment on Season 3 – more than 3,000 shots over eight episodes – Worth realized early on that he needed to concentrate on city environments and robots. “So we said, ‘Which hard-surface company do we want to use and which city environment do we want to use? I actually got reels from all the different vendors that I worked with and did a ‘blind taste test’ with Jonah [series co-creator Jonathan Nolan]. Thankfully, we were on the same page with who we wanted to go with.”

In the end, Pixomondo was chosen to carry out environment VFX, with DNEG handling the new robots. Other vendors on the show included CoSA, Important Looking Pirates, RISE FX, Crafty Apes, Deep Water/Yafka and Bot VFX.

Visual Effects Supervisor Jay Worth. (Image courtesy of Jay Worth) (All images copyright © 2020 HBO except where noted.)

“Every inch of every frame is designed. You can’t really point the camera anywhere in the [Westworld] world without [visual effects], the art department or costume having to touch it. Even when we were back in the park, we had to remove everything in the sky. We’d be up on a hillside and say, ‘Oh this is great!,’ and then all of a sudden you see a random telephone pole in the background. Those times happen a lot where we’re at now this season.”

—Jay Worth, Visual Effects Supervisor

An airpod appearing in Episode 1 of Season 3. Pixomondo handled city landscapes this series.

Aaron Paul as Caleb Nichols with the construction robot George. DNEG was responsible for Season 3’s various robots.

Westworld co-creator Jonathan Nolan with Lena Waithe, who plays Ash in Season 3.

“We always built everything within the park with a very clear understanding of what the outside world looked like. If you think about it, the park is set up for the uber-rich to come and figure out how to play. So we thought, ‘What does the world look like if the .01% have been able to have their way more than is probably healthy?’”

—Jay Worth, Visual Effects Supervisor

An airpod flies through neo-Los Angeles in an Episode 1 VFX shot crafted by Pixomondo.

Bernard (Jeffrey Wright) stands with a drone host in a scene from Episode 2 of Season 3.

VISUALIZING THE OUTSIDE WORLD

Since the beginning of the series, there were many discussions between Worth and Nolan about how outside the park would appear, Worth says. “We always built everything within the park with a very clear understanding of what the outside world looked like. If you think about it, the park is set up for the uber-rich to come and figure out how to play. So we thought, ‘What does the world look like if the .01% have been able to have their way more than is probably healthy?”

Inspiration for that look and feel came from locations that tended to already have a futuristic presence, including Singapore and the City of Arts and Sciences in Valencia, Spain. Production filmed at those two cities, and then final scenes would be a mix and match between them and Los Angeles, with visual effects led by Pixomondo carrying out the merger.

While there would be several hero environment shots, a large component of the backgrounds also included readying them for use on LED wall projections for use during ‘airpod’ flying scenes and other sequences. That approach was something that had been attempted, but not fully developed for previous seasons.

“Jonah and I have always been passionate about projection and LED technology,” states Worth. “That goes back to the park map, and just generally moving away from throwing up a bluescreen or greenscreen. With the progress in LED technology, as well as real-time tools like Unreal Engine, we really wanted to embrace that new approach.

“We have tests from four years ago,” adds Worth, “where we built our own little supercomputer and had projectors and built a real-time environment. And we put a sensor on the camera and sensors around the room to see how it worked to have it translate to camera. Unfortunately, at the time we just couldn’t quite get it there.”

On Season 3, LED screens and real-time rendered backgrounds were utilized for flying scenes, allowing VFX shots to be captured in-camera.

The Westworld production took on more of a partner approach this time around for the LED screens, collaborating with Epic Games (the makers of Unreal Engine), Fuse Technical Group for LED screens, and Profile Studios to handle the on-set calibration and operation of screens and imagery.

The screens first came into use for the airpod flying shots. Production designer Howard Cummings’ art department produced the sleek airpods, designed by concept designer Thang Lee. Special effects contributed a gimbal rig that allowed the actors to be filmed inside the airpods and go through banking and other moves. This was filmed against a large semi-circle LED wall, onto which were projected augmented helicopter plates of Los Angeles.

“It meant we were really able to put the camera wherever we wanted and get real-time playback, real interactive light and real reflections – all those things that make it really feel like these people are flying,” says Worth. “In fact, we basically built a mini-amusement park ride. The actors are in this world and they’re looking around and they really feel like they’re flying through the city.”

While extra reflections were composited into the windows, along with additions to the practical water droplets, most of the views outside were achieved in-camera. This particular setup did not require real-time rendering, but other scenes did utilize that approach.

“There’s a sequence in Episode 3 with Charlotte Hale (Tessa Thompson) in her office at Delos,” advises Worth, “where we built the entire asset in Unreal. All of those shots are with the massive 150-foot LED wall outside a piece of glass, so we had to test what shooting through glass would involve, making sure that there were no aberrations and that it translated correctly to camera.”

Existing scenes of Los Angeles were augmented digitally to include new buildings, as well as the flying airpod.

ROBOTS RISING

Westworld has already seen its fair share of robots for the park’s ‘hosts,’ although of course those are of the kind that look almost like humans. In the ‘real’ world, robots were envisaged as metallic-surfaced versions, i.e. more like the classic robots we might come to think of.

“One of the challenges we had on the show was actually creating those new robots who are standing right next to these humanoid robots that we’ve designed that obviously are so much more human than anything we’ve ever seen,” identifies Worth. “It was somewhat of a weird dichotomy – how to create a robot that you’ve not quite seen before but still feels like a more utilitarian robot that’s still next to one of these robots that’s so human and lifelike. It was a fun challenge.”

To help bring the robots to life the production at first considered whether to build them practically, but ultimately went down a motion capture route. “We got our suits and got our motion capture system set up for being on construction sites and different places and got really good performers to be these robots for us. It ended up being a process where we used some of the motion data acquired on the shoots, some hand-animation, and then DNEG’s team did some separate capture during post.

LED screens became a larger part of the filmmaking process for Season 3.

Architecture at the City of Arts and Sciences in Valencia, Spain informed final shots.

Exterior of Marina One in Singapore serving as a plate for an explosion shot.

A central theme in Westworld imagery over three seasons has been the 3D printing of hosts.

“We took a lot of what the performers did,” continues Worth. “Then there’s definitely times when it’s hand animation, and there’s definitely times where DNEG and their crew jumped right in and did some additional motion capture for us. I’m really happy with how they all turned out.”

ADDITIONAL EFFECTS

Westworld is one of those series that might be considered to have a set of principal visual effects (such as the environments and robots), while also including a plethora of other supporting VFX work.

For example, says Worth, “every time you see a tablet or a phone, we have to burn it in. We have an amazing graphics team that design all these wonderful things. Anytime you see a screen, we do try and shoot as many things practically and then we either replace some of it or all of it. We want our screens to be very thin, so the actors tend to be just holding a piece of metal.”

A number of the vehicles in the ‘real’ world were built props, like the airpods, which could be suspended on a crane and be filmed coming to the ground or put on the gimbal as described above. There was also a rideshare vehicle that could actually drive.

Then there were augmentations made to live-action vehicles, such as the autonomous motorbike seen this season. “The motorcycle is a fully operational designed electric motorcycle that is custom-built,” explains Worth. “We ended up painting out things when it’s automated. There’s a rider in a black suit on it for the shoot, which we paint out. We also did a couple of modifications to remove a chain that felt a little too antiquated, and did a little bit of touch-up work. But it’s a real thing there in the frame.”

THE WESTWORLD ‘LOOK’

Westworld is shot on film which, suggests Worth, aids in bringing a distinctive look to the show. It also brings its own set of challenges for the visual effects crews.

“The technologies that we utilize now, they just lean so far into digital,” he notes. “Simple things like pulling keys can get a lot more challenging on film than they can on digital. And of course we have to go through film scanning and all that kind of thing.”

Still, film provided the show’s creators with a surprising benefit in terms of the LED wall sequences. These were actually components that many thought should be tackled with digital cameras. “But,” recalls Worth, “Jonah felt that doing those scenes on film was just going to actually end up looking better, that it was going to get a softness and a cinematic quality that was really beautiful. And it turned out to be right.

“We shot tests on both and it’s still shocking to me how good the LED walls look on film compared to digital. They just end up looking so much more beautiful and cinematic. The same reason why we like shooting on film for our actors and the way you just feel how light looks more beautiful. It really ended up being to our advantage to shoot it all on film.”

“Every time you see a tablet or a phone, we have to burn it in. We have an amazing graphics team that design all these wonderful things. Anytime you see a screen, we do try and shoot as many things practically and then we either replace some of it or all of it. We want our screens to be very thin, so the actors tend to be just holding a piece of metal.”

—Jay Worth, Visual Effects Supervisor