Super Misfits: The VFX of THE UMBRELLA ACADEMY

By IAN FAILES

By now, we’re all used to superheroes on the big and small screens, but The Umbrella Academy – the Netflix series based on the Dark Horse comics and developed by Steve Blackman and Jeremy Slater – follows seven children all born simultaneously, who each seem to have mysterious superpowers, thrust together into a dysfunctional family.

Still, with superpowers come super visual effects, and The Umbrella Academy has many, including the aftermath of a major future apocalypse, spatial and time-travel jumps, the destruction of the moon, and a fully CG character named Pogo, a chimpanzee who can speak. VFX Voice asked the show’s Production Visual Effects Supervisor, Everett Burrell, and Visual Effects Supervisor Chris White from Weta Digital – which created Pogo – about these significant sequences and characters.

THE PATH TO POGO

In the series, Pogo is an assistant to Sir Reginald Hargreeves (played by Colm Feore), a billionaire who has adopted the seven children into his ‘Umbrella Academy.’ To create an advanced chimpanzee, Burrell knew he and his team would need to craft a creature that could talk, wear a suit and interact in scenes with live-action actors. That initially led him to consider a ‘man-in-suit’ option for shooting scenes, perhaps to be augmented with a CG head replacement.

However, Burrell then consulted Joseph Conmy, Senior Vice President Visual Effects at 20th Century Fox, about his work with Weta Digital on the Planet of the Apes films, where Weta had very successfully realized the primates entirely in CG. It was a daunting task for such a crucial character to be created on what are usually shorter schedules for TV, but Burrell pushed the production on going fully digital with Pogo.

“Netflix liked the idea of pushing the envelope on doing an all-CG character for television, especially for a show like this where it had to be very subtle and very supporting as a character. He wasn’t a dragon, flying around shooting fire. He had to simply be in the room and have conversations. It was a few months of terror, but once we saw the first test from Weta Digital, I think everybody calmed down.”

On set, actor Ken Hall stood in for Pogo wearing a gray tracking marker suit. Two 4K cameras off to the side of the main Alexa 65 camera acted as witness-cams to provide environment and body reference for Weta Digital. “Then we would pull Ken out and do clean plates, and then I would do HDRI capture with a gray ball,” says Burrell. “We also had a really great stand-in Pogo head with hair punched and painted so the director of photography could light it specifically for the shots.”

The voice of Pogo was provided by Adam Godley, who was also filmed performing the lines so that his expressions and emotions could be inputted into the CG character. Weta Digital keyframed the final animation, but also used its own motion-capture stage to re-create Pogo’s actions based on the performances of Hall and Godley.

Pogo unfortunately meets an untimely end at the hands of Vanya (Ellen Page). Weta Digital had a significant hand in how the death scene occurred. “We were able to send the production some previs that we had done showing how Pogo gets picked up and thrown against these antlers,” says White. “We also did some tests at Weta where we took some fabric from different shirts and practical blood and showed how the blood would come out and how it would form. We even had some antlers that were from another show that we pushed through.”

For Pogo’s suit itself, Weta Digital relied on a physically-based cloth model. This model replicated the technique of spinning and weaving to make textiles the same way they are made in the real world. “It’s a new technique developed where we could go down to the individual thread, but it didn’t have to be hand modeled,” notes White. “You could give it different fabric and weave models and it does all the correct lighting, but it does it procedurally within the system itself.

Ken Hall in a capture suit films a scene with Ellen Page as Vanya. (All images copyright © Netflix 2019.)

Weta Digital tracked the performance and replaced it with their CG Pogo.

The final rendered shot.

“Netflix liked the idea of pushing the envelope on doing an all-CG character for television, especially for a show like this where it had to be very subtle and very supporting as a character. He wasn’t a dragon, flying around shooting fire. He had to simply be in the room and have conversations. It was a few months of terror, but once we saw the first test from Weta Digital, I think everybody calmed down.”

—Everett Burrell, Production Visual Effects Supervisor

“The other thing with the clothing,” adds White, “was that originally we had thought of fitting this clothing to Pogo, but then we talked to Everett about it and he said, ‘Well, the public doesn’t know about him, so it’s not like he’s going to go out and get fitted or anything like that, so it’s okay for his jacket to not fit exactly right.’ You’ll see in certain scenes that it doesn’t hang like a well-fitted jacket and his pants are a little bit off.”

Plate photography for a scene showing the apocalypse environment made use of partial sets and greenscreens.

The final apocalypse shot extended the carnage with extra buildings, fire and smoke.

Number Five investigates the destruction in this original plate.

Final shot with visual effects augmentation.

“In one of our camera tests, [actor] Aidan [Gallagher] was in a hallway, and he was just kind of goofing off in between takes. I asked him, ‘Can you do a little hop in the air? Can you do a little jump?’ And he did a little jump, and then I said, ‘Well, that’s actually kind of neat, and it will help motivate you when you teleport, so why don’t you run towards camera, jump in the air, and then just walk out of frame?’”

—Everett Burrell, Production Visual Effects Supervisor

BRINGING THE APOCALYPSE

Before we actually find out just what causes the apocalypse, the viewer sees glimpses of it from the future, courtesy of Number Five (Aidan Gallagher), who is able to travel through space and time. Only a small portion of the destroyed apocalyptic environment was built, says Burrell. “There was a pile of rubble, the façade of the Umbrella Academy home and maybe a couple other buildings. The way it was designed was like an amphitheater of rubble, so when you were at the bottom of the amphitheater and you looked around, the rubble covered the skyline. But then occasionally we’d want to go high and wide to reveal the apocalypse.”

That extension of the destruction was handled by Spin VFX, which took Lidar and photographic reference of the set and built extra derelict structures and general mayhem. “We also had great concept art from production designer Mark Worthington,” adds Burrell. “It really set the tone. Every time we had an incoming director, we gave them this packet which we called the ‘director’s welcome pack,’ and it had all the concept art, and all the R&D and tests that we had done that had been approved.”

JUMPING WITH NUMBER FIVE

In order to show us the apocalypse, Number Five jumps forward in time. He can also do much smaller spatial jumps. For these, the visual effects team started with some early R&D. “In one of our camera tests,” recalls Burrell, “Aidan was in a hallway, and he was just kind of goofing off in between takes. I asked him, ‘Can you do a little hop in the air? Can you do a little jump?’ And he did a little jump, and then I said, ‘Well, that’s actually kind of neat, and it will help motivate you when you teleport, so why don’t you run towards camera, jump in the air, and then just walk out of frame?’”

Spin took that footage and iterated on several kinds of spatial jump effects, all the way from heavy distortion to subtler images. The final effect – dubbed ‘jelly vision’ – appeared, describes Burrell, “as if you’re pushing your hand through a jelly membrane, just for a few seconds, and then it pops. It’s really, really subtle, but you get a little bit of texture, you get a little bit of striations, almost like the universe is bending as he does his spatial jumps.”

Time-jumps by Number Five were more elaborate in nature, since they open up a portal that appears more like a displaced and distorted gateway. In addition, it was decided to have the portals ‘turn off’ like an old black and white television set. This was realized by having the portal “sucked into one little point with a little residual, horizontal lens there,” outlines Burrell.

MOON CRUSHING

In the final episode of the season, the cause of the apocalypse is uncovered – the suppressed powers of Vanya are finally unleashed, and her fiery abilities burn a hole in the moon, which begins to break up and rain down on the Earth. Here, Burrell relied on several VFX vendors to deliver shots of the chaos, with matte paintings and digital environments shared between Spin, Folks VFX, Deluxe and Monsters Aliens Robots Zombies (MARZ).

“We also brought in Method Studios to do all the really big destruction of the moon and of the city,” says Burrell. “They were heavily involved in all the effects animation and the R&D of blowing up the moon and the beam that Vanya shoots out from her chest. That was very elaborate, and I think we only had six weeks to make all that happen, but Method really came through, destroying the moon. That stuff looked great.”

UNEXPECTED EFFECTS

Among the complex challenges of CG characters, time-traveling effects and apocalyptic environments, The Umbrella Academy also required some largely invisible VFX when some undesirable locations filmed during mid-Winter were actually suitable once Spring had hit.

“The locations that Steven Blackman had originally hated in Winter became these beautiful, green, lush tree areas,” notes Burrell. “So we had to go back to the first five or six episodes and add trees and leaves to every shot that did not have them in the wintertime. We went back to every location, and we shot with our 4K cameras those same angles once the trees had bloomed. I gave those to Folks VFX, and they basically comp’d them in, and when we couldn’t do that, we’d do some CG trees.

“We were very lucky that we had the time and the patience to go back and get all those angles,” adds Burrell. “I’m almost prouder of that than Pogo because no one will ever know.”

Scenes of Luther (played by Tom Hopper) on the moon first required a greenscreen shoot.

CG elements were then added into the frame.

A final moon composite.

Some scenes were shot in Winter when leaves had fallen off the trees.

Visual effects replaced the leafless trees with lush green ones to match other plate photography