Weta Digital: Moving Cities and Bringing Anime to Life

By TREVOR HOGG

In Mortal Engines, London consists of eight tiers, each supported by a suspension system that results in a smoother ride for the upper classes that live in higher levels of the city. (Image © Universal Pictures and MRC)

Pushing creative and technical boundaries is a mantra for Weta Digital, which had to reimagine London, England as a massive urban predator travelling across Europe in Mortal Engines, and in the process had to produce an entirely new pipeline.

“With every project there’s always something different,” notes Animation Supervisor Dennis Yoo, who traded the biologically enhanced primates of War for the Planet of the Apes for a futuristic world where cities literally hunt and dismantle each other. “In Mortal Engines, it was literally the scale of everything. We built these moving cities to actual scale in CG. The math involved got a bit crazy. The happy spot for the software is for x, y and z to be set at zero. When these cities move a few kilometers away we started to lose accuracy, so they had to be cheated back to zero space.”

Pipelines needed to be revised in order to accomplish the work. “There were some growing pains on how we were going to animate these cities,” states Yoo. “We developed a layout puppet.” A city was divided into different tiers consisting of sections that could be moved independently from each other. “There’s this hierarchy structure in the puppet. You can go, ‘I need this part of London.’ You could load just that part of London. Unfortunately, another aspect of that system was if you only wanted a gun situated in a particular part of London you had to load that entire layout.” Different teams on the film had to collaborate in order to resolve issues. “We got the creature lead to sit with the animation team so we could troubleshoot things on the fly with each other. Later we had the layout lead as well. It was the only way that we could communicate properly about something that complex.”

Previs Supervisor Marco Spitoni has been working on Mortal Engines for a decade. “Some of the previs was on point where we literally grabbed the previs camera and put that into the correct world space,” remarks Yoo. “Also, [producer] Peter Jackson [King Kong] went onstage, shot a bunch of virtual cameras, and we moved those into our scene files. We did try to streamline it where we could work in a 3D environment. We solved a lot of issues by using a tech called Koru Skinning which does calculations outside of Maya, and in such a way that is easier and faster for the animation software program to distinguish and organize.” Concept art developed by Nick Keller established the look of the predator city of London. “The bottom tiers were industrial-type buildings, and the further you move up, the posher and sleeker the city becomes. The concepts for that were there right from the beginning. St. Paul’s Cathedral, which contains the Medusa weapon, was switched around.”

Atmospherics like fog were incorporated into the shots to add a sense of depth. “No one wants to see a snail race,” laughs Yoo. “Initially, it was thought that the cities would move a maximum of 100 km an hour [62 mph]. In the trailer, to make things more dynamic, London was sped up in the edit. When trying to recreate it we realized that London was going Mach 2. We had to figure out how to make it more consistent so the effects team didn’t have to do a bespoke simulation for every shot. We tried to keep London within the 300 km mark, which is still unbelievable, but looks exciting.” It was fun animating the cities. “There is a cool tiny city called Saltzhaken that gets eaten up by London right at the start.”

A key trait for the character of Shrike in Mortal Engines is his glowing green eyes. (Image © Universal Pictures and MRC)

Not all of the cities in Mortal Engines are landbound such as the aerial dwelling of Airhaven. Volumetric clouds were significant in conveying the size and scale of Airhaven. (Image © Universal Pictures and MRC)

In Mortal Engines, the matte painting department 3D-printed treads from the city models, marked up model clay with them, and did a photometric scan to create a pallet of displacements that they could put into the ground. (Image © Universal Pictures and MRC)

A fleshy look was incorporated into the metallic face based on the facial features of actor Stephen Lang to enable Shrike to emote. (Image © Universal Pictures and MRC)

The final shot of the opening chase sequence in Mortal Engines, with clouds and lighting. (Image © Universal Pictures and MRC)

Another significant animation challenge for Yoo was the CG character of Shrike (played by Stephen Lang), a deadly warrior and human-machine hybrid. “The performance of Stephen Lang on set was constantly changing as director Christian Rivers and the producers were trying to figure out what they needed from the character. Initially, Shrike was literally a jaw and a skull inside a metal face. Later we decided to do a full facial rig to pull some emotions out of his face.” A distinguishing trait are the character’s glowing green eyes. “Shrike wasn’t hard to rig. The only things that he had extra of were his feet, which are like claws, and the ability to flick out his fingernails.” The physical motion was simplified to be more humanoid to avoid Shrike looking like a guy in a mocap suit. “One added thing that we didn’t expect to do was his flowing jacket, which the client wanted to see in our animation, so the cloth simulation had to be baked in.”

While Mortal Engines was being developed, Weta Digital was also involved in producing a believable photorealistic anime protagonist for Alita: Battle Angel, which is based on the cyberpunk manga by Yukito Kishiro that revolves around an amnesic cyborg with lethal martial arts skills. Alita was one of the most ambitious CG characters developed at the six-time Oscar-winning Wellington, New Zealand-based visual effects facility, which has previously created Gollum (the Lord of the Rings trilogy), Neytiri (Avatar), Caesar (the Planet of the Apes trilogy) and Thanos (Avengers: Infinity War). “She has an anime style with those big eyes, but we put a lot of energy into making sure the puppet was going to capture all the subtleties and complexities of Rosa Salazar’s performance,” explains Animation Supervisor Mike Cozens, who previously helped to bring to life the title creature in Pete’s Dragon. “This film was the first time that we’ve done a direct-match actor-puppet before putting the motion onto the actual CG character puppet where it is furthered refined.” There was also a movement away from FACS (Facial Action Coding System), which is a method of mapping out muscle motions in relation to emotional facial poses developed by psychologist Paul Ekman. “That’s how we’ve been building puppets over a number of years. The FACS puppet build process tends to build for extreme facial expressions [for example: extremely happy, angry or sad]. However, people tend to be quite efficient with their lips when talking, and there’s a whole bunch of complexity that happens in the subtle range. We had to rethink and rework a bunch of posing in order to catch those speech phonemes.”

In Alita: Battle Angel, Houdini was used to procedurally model proper irises with each eye consisting of a million polygons. (Image courtesy of Twentieth Century Fox)

Zapan consists strictly of the face of Ed Skrein with everything else including the back of his head replaced with CG in Alita: Battle Angel (Image courtesy of Twentieth Century Fox)

“[Alita] has an anime style with those big eyes, but we put a lot of energy into making sure the puppet was going to capture all the subtleties and complexities of Rosa Salazar’s performance. This film was the first time that we’ve done a direct-match actor-puppet before putting the motion onto the actual CG character puppet where it is furthered refined.”

—Mike Cozens, Animation Supervisor, Alita: Battle Angel

Dr. Dyson Ido (Christoph Waltz) recovers the Cyber Core of Alita. (Image courtesy of Twentieth Century Fox)

“We worked hard to get the correct lip behavior in relation to the teeth and gums. All of this structural work was expanded upon, reworked, and put into the puppet. With that and the phoneme work, the detail of the facial performance came together and we had a photoreal facial performance.”

—Mike Cozens, Animation Supervisor, Alita: Battle Angel

Alita examines her Cyber Girl body made out of alabaster, which was originally constructed by Ido for his daughter. (Image courtesy of Twentieth Century Fox)

Alita prepares to take part in the lethal gladiator sport of Motor Ball. (Image courtesy of Twentieth Century Fox)

A two block set was constructed of Iron City in Austin, Texas which still required digital augmentation to get necessary scope of the environment. (Image courtesy of Twentieth Century Fox)

Rosa Salazar, wearing a mocap suit and facial markers, speaking to filmmaker Robert Rodriguez and co-star Keean Johnson on the set of Alta: Battle Angel. (Photo: Rico Torres)

Then there was the matter of the character’s facial structure. “The jaw is the primary bone in your face, so if that motion is incorrect everything goes askew from there,” explains Cozens. “We went back and looked at how the jaw was pulling the muscles and flesh around on the face, how the mouth structure changed and how the lip corners behaved in relation to jaw motion. The other thing that we found was that the inside of the mouth, most of which you don’t see, is a critical part of the face build; this included the inside of the lips and the modiolus [a point at the corner of the mouth where eight muscles meet]. We worked hard to get the correct lip behavior in relation to the teeth and gums. All of this structural work was expanded upon, reworked, and put into the puppet. With that and the phoneme work, the detail of the facial performance came together and we had a photoreal facial performance.” Real-time renderer Gazebo was utilized to prerender the animation with a lighting pass that included shadows. “The lighters provide the final scene lighting so we have key and fill directions correct, so when the animation evolves the lighting is not jumping all over the place. Having the final lighting setup at the animation stage also helped us to debug problems with emotional performance.”

Unlike Alita, who is entirely CG, a few cyborgs have live-action faces and digital bodies or live-action partial bodies and digital partial bodies. “Because the film was shot native stereo, all of that stuff had to be bulletproof,” remarks Cozens. “All of the match moves and camera tracks had to be tight in order to make the CG and the live action stick together. Often when you’re doing a character like Zapan (played by Ed Skrein), who has a live-action face on a CG body, we tend to be quite limited with what we can do with the live-action part of the final composite. It can’t move. A couple of the animation R&D guys came up with a helpful new way of doing cards where we could move the element around in-camera without breaking the stereo or its connection to the CG. That gave us more freedom to recompose and augment performance for those characters when we needed and still have the result be believable. Developments like that opened up our ability to get the shots that director Robert Rodriguez wanted.

“Rosa’s performance evolves through the film as Alita finds her warrior spirit and discovers what it means to be human,” notes Cozens. “The stunt team, led by Garrett Warren, had done a lot of development for the on-set stunts while, at the same time, we worked through a bunch of previs for some of the larger CG scenes. I had an opportunity in Austin, Texas to meet with Garrett and fight coordinator Steven Brown to align what we were doing in terms of her motion studies.”

Alita’s abilities move quickly from an ordinary girl to being able to do extreme things, but all of that needed to be grounded in real-world physics. “The interesting thing about Rosa’s performance is that it’s multilayered, and you can see that she’s saying one thing and thinking another. There’s an inner monologue that’s happening. Catching this type of complex performance on Alita while making sure that she looks good was the biggest challenge and the thing that I’m most excited to see onscreen. As Robert said, ‘Alita is a living manga character who quickly becomes more than human.’”