By IAN FAILES
By IAN FAILES
The holy grail in visual effects is often said to be digital humans. A quick look back at the last few years has seen some amazing achievements in the field – Weta Digital’s Paul Walker for Furious 7, MPC’s digital Arnold Schwarzenegger in Terminator Genisys, ILM’s Governor Tarkin for Rogue One and Image Engine’s Wolverine in Logan.
Another leap has now been taken with Denis Villeneuve’s Blade Runner 2049, in which overall Visual Effects Supervisor John Nelson and MPC combined to produce a digital version of Rachael, the replicant played by Sean Young, who Rick Deckard (Harrison Ford) fell in love with in the original Blade Runner film released in 1982.
Needing to depict Rachael exactly as she appeared in that earlier movie, the filmmakers settled on crafting a CG head for the character, based on scans and reference of Young, a stand-in on set, and MPC’s digital artistry. VFX Voice went step by step through the process with Nelson and MPC Visual Effects Supervisor Richard Clegg.
“It’s one thing to make a digital human look real, but it’s another thing to make them act real.”
—John Nelson, Visual Effects Supervisor, Blade Runner 2049
Nelson knew that recent advancements in CG humans had been significant, and so he was confident MPC could pull off the shots. But he thought there should be an extra step in making ‘Rachael Reborn,’ and that Deckard and the character share some kind of ‘real’ connection on the screen.
“It’s one thing to make a digital human look real, but it’s another thing to make them act real,” suggests Nelson. “Which means we really got into the performances of Sean as a young woman and what were her little tells. Then we tried to incorporate them into Rachael Reborn. As she comes out, for example, we felt it was really important that she is the superior, beautiful fashion model replicant. You want to echo back to when Deckard first meets her 30 years ago. Then, as she gets up to him, Denis said, ‘I want it to feel like it’s somebody who loves someone but he hasn’t seen them for 10 or 15 years. They see them in a train station. Like you just can’t stop your emotions.’”
MPC embarked early on a detailed reference-gathering exercise for time-appropriate footage of Sean Young. Obviously, this included Blade Runner, but Dune was another useful film from the era. A plaster cast had been made of the actor from that time, but had deformed and so it just became part of the reference process.
Young and stand-in actor Loren Peta both participated in Light Stage scans at USC Institute for Creative Technologies. “For Sean,” says Clegg, “we knew we could use the scan for any key proportions that really don’t change with time, like brows or jawline. If you get the skeleton right and the skull, then you know that you’re on the right track. You can try to figure out the line-ups based on the old film footage, but the lenses and camera set-ups then did not come with any data and so we had to reverse-engineer that side of things.”
The actors carried out a Facial Action Coding System (FACS) capture for different face shapes, and Peta’s acquired skin textures were also used to inform the final look in the CG head, including things like the right level of peach fuzz and any facial imperfections.
At this point, though, Rachael goes from confident to longing. She touches Deckard and he rejects her. The emotional changes and eventual sadness had to be reflected in the CG actor in the brief scene – a major challenge for the visual effects team that they would need to replicate. “If you notice when she gets rejected she gets a little tear in her eye,” points out Nelson. “Just for a moment. It’s almost imperceptible but it’s there.”
Armed with the scans and facial animation data, MPC got to work on some test shots. They took scenes with Rachael in them from the original movie and used them as a benchmark for their CG head. “We did three shots from the original movie to try and convince everyone that we had it in the bag and we knew exactly what we were doing,” outlines Clegg.
“To begin with, we just showed our shots side-by-side with the original footage, but then one day I surprised everyone by cutting in our composites of digital Rachael with the rest of the sequence so that we could play a game of, ‘Guess which shot is the CG shot?’” (Clegg says Nelson always guessed correctly, but when he had to second guess himself, everyone knew they were on the right track).
On set in Budapest, Peta performed the role sporting the appropriate clothing and hairstyle, with some tracking dots painted on her face. An early challenge Clegg identified was that Director of Photography Roger Deakins, ASC, BSC, CBE had lit the vast set with caustic lighting affected by water ripples. Although that ultimately did not prove too troublesome, a large ring of lights above the set controlled by an iPad meant the lighting would change constantly. The VFX team set up several timecode-sync’d witness cameras and recorded as much on-set reference as possible to help replicate the lighting.
Peta and Young also participated in a videogrammetry facial-capture which involved a further FACS shoot while they ran through their lines. “They watched back the footage of Loren and Harrison from the shoot and then they would re-track and re-perform that,” says Clegg. “That became a really good guide for making sure we kept Rachael’s likeness.”
“As [‘Rachael Reborn’] comes out, for example, we felt it was really important that she is the superior, beautiful fashion model replicant. You want to echo back to when Deckard first meets her 30 years ago. Then, as she gets up to him, Denis said, ‘I want it to feel like it’s somebody who loves someone but he hasn’t seen them for 10 or 15 years. They see them in a train station. Like you just can’t stop your emotions.’”
—John Nelson, Visual Effects Supervisor, Blade Runner 2049
“To begin with, we just showed our shots side-by-side with the original footage, but then one day I surprised everyone by cutting in our composites of digital Rachael with the rest of the sequence so that we could play a game of, ‘Guess which shot is the CG shot?’”
—Richard Clegg, Visual Effects Supervisor, MPC
“We found about 50 to 100 photographs, some of them were photographs of Sean Young delivered to us from her early 20s, and then the original film footage, and we set up a big line-up scene with all these frames. We had an automated process that every day when the modeler had finished their day, they basically released their model and it would just pump out a quick render, which we compared frame by frame with our model next to the head.”
—Richard Clegg, Visual Effects Supervisor, MPC
In crafting the digital model of Rachael and in texturing and rendering that model, MPC’s artists had to deal with a character that had very soft and subtle features, delicate dimples in the cheeks and definitive contours. Since the reference of Young was slightly different in each photo or piece of footage, Clegg’s approach was to do line-ups.
“We found about 50 to 100 photographs, some of them were photographs of Sean Young delivered to us from her early 20s, and then the original film footage, and we set up a big line-up scene with all these frames. We had an automated process that every day when the modeler had finished their day, they basically released their model and it would just pump out a quick render, which we compared frame by frame with our model next to the head.”
That facilitated sessions between Nelson and Clegg where they would continually massage the model by re-visiting the line-ups. “We did the same thing for her hair and groom,” says Clegg. “We picked some key shots we wanted to base it on because obviously Rachael’s got quite an intricate hairstyle, which also changes shot to shot in the original film.”
Rachael Reborn’s eyes were also crucial to the final character’s performance. “I’d say to the team about 10 times a day, ‘The most important thing is the eyes,’” recalls Clegg. “And it’s interesting how the eyelashes play a big part; if you widen them just slightly you get more heavy-set shadows from the lids and things like that. So we would nudge things and maneuver things around to get it all as physically accurate if possible.
“We ended up animating the whole face, so there were a lot of facial shapes and a lot of sculpting going on, not just on the base model, but also on all the face shapes,” adds Clegg. “When we were building the face shapes as well as doing line-ups to her head, we’d also had to find lots of takes from the original movie with similar motion that we wanted, and we based our shapes on a lot of that as well.”
With the rendering, which was done in RenderMan, Clegg says that although there were constant adjustments to CG Rachael, “it was all based on studying how it’s working in real life and nature. So when we put it in our shot and lighting context, we got accurate results. I think with this layer of subtlety on a digital human, you’ve got to get more into the science and just try and understand the physics of it, and be as precise as possible.”
Still, there was a level of imperfection built into digital Rachael’s face, again aiming for the most realism possible. Clegg says he had to ensure artists understood the head as rendered cleanly with extra sub-surface scattering would look slightly different once in the scene lighting. “I remember when we were looking at it in dailies and you render, and they’d just look at me and say, ‘Is that it?’ Because in their mind they were basically painting a zombie, and then you put the model in the shot and it has a softer lighting, and the scattering and the effect of that just disappears.”
“I’d say to the team about 10 times a day, ‘The most important thing is the eyes.’”
—Richard Clegg, Visual Effects Supervisor, MPC
The final result is a stunningly real re-creation of Rachael as she was in 2019 (i.e. Sean Young in 1982). Nelson is particularly proud of the emotions Rachael goes through and the very small details, including the rendered flyaway hairs. He says that Villeneuve also began to treat the CG character as a real actor in the film.
“At one point I said to Denis, ‘I can redo this part so her expression is exactly the same from this shot to this shot.’ But he goes, ‘No, no, John, it’s just like directing a real actress, you know? Actresses, when you go from this shot to this shot, their expression can change slightly because they’re actresses. It’s just like directing a real actress.’ I feel very good about that. But it was definitely one of the hardest things we did.”