VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


June 04
2019

ISSUE

Web Exclusive

How Framestore Fleshed Out ‘Smart Hulk’ for AVENGERS: ENDGAME

By TREVOR HOGG

Machine-learning software was trained to capture micro-moments that made for a more believable facial performance. Front is Chris Evans, who plays Steve Rogers/Captain America. (Images courtesy of Framestore and Walt Disney Pictures/Marvel Studios.)

Along with ILM, Framestore was responsible for creating Smart Hulk, a peaceful coexistence between scientist Bruce Banner and his volatile, green alter ego that makes his first appearance in Avengers: Endgame.

“ILM started developing Smart Hulk probably during Infinity War,” states Framestore Visual Effects Supervisor Stuart Penn. “Before the beginning of Endgame they turned over the base model, textures and some of the face shapes. From that, we developed our own muscle system, added to the face shapes, built our own face rig, and were provided reference by the client of turntables and facial workouts that ILM had done to make sure our facial performances were matching.”

Stuart Penn, VFX Supervisor, Framestore. 

Smart Hulk was the most complex facial rig created by Framestore. The two key areas to translate were the movement of Mark Ruffalo’s mouth and the way his eyes moved. 

 

“It was down to the level where [Mark Ruffalo] has specific muscle structures in his face, and it was translating those movements into Hulk. The two key areas were the movement of his mouth and the way his eyes move. It was the essence of Mark’s performance coming through on quite a different face.”

—Stuart Penn, Visual Effects Supervisor, Framestore

Machine-learning software was trained to achieve certain keyframes in specific shots.  “That allowed us to generate a quick first-pass animation for the entire sequence,” explains Penn. “We would quickly render it out and send it back to the client so they had something quickly for their edit. We then moved to keyframe animation on the facial stuff to get what we weren’t getting from the solves. The solves also gave us a micro-movement pass that we could extract from the machine-learning data.”

Because Smart Hulk is a hybrid of Hulk and Bruce Banner, there was a heavier reliance on the performance capture of Mark Ruffalo. “It was down to the level where he has specific muscle structures in his face, and it was translating those movements into Hulk. The two key areas were the movement of his mouth and the way his eyes move. It was the essence of Mark’s performance coming through on quite a different face.”

Smart Hulk is slightly less green in Endgame, as the desire was to have a more human subsurface skin coming through underneath. At right is Scarlett Johansson, who plays Natasha Romanoff/Black Widow, and left is Chris Evans.

“It was the most complex facial rig that we’ve ever done. It had many more shapes and additional controllers for the animators to get that fine level of performance, and then all of the dynamic stuff and machine-learning micro-movement that we then plugged in on top.”

—Stuart Penn, Visual Effects Supervisor, Framestore

“It was the most complex facial rig that we’ve ever done,” continues Penn. “It had many more shapes and additional controllers for the animators to get that fine level of performance, and then all of the dynamic stuff and machine-learning micro-movement that we then plugged in on top. The body rig was similar to other rigs that we’ve done before, but we didn’t have a full, working muscle system in there, which we needed.  Marvel was keen that he’s muscly and quite bulky, so when he moves and flexes his arms you actually see those muscles moving and sliding underneath the costume. We had to overdo the muscles to read them through the cloth of the costumes.”

Smart Hulk needed to be able to convey a wide range of emotions. “The hardest thing was that he was doing comedy performances quite a bit. Comedy is all about timing and subtle changes in expressions. We would always go back to watch what Mark did to see what it was about his performance told you what his emotional state was.”

A more finesse approach was adopted for the animation of Smart Hulk since he is capable of graceful and delicate movements. At right is Chris Evans.

Smart Hulk (Mark Ruffalo) was rigged to believably handle a wide range of emotions, including comedy, like in this moment with Chris Evans.

“Marvel was keen that he’s muscly and quite bulky, so when he moves and flexes his arms you actually see those muscles moving and sliding underneath the costume. We had to overdo the muscles to read them through the cloth of the costumes.”

—Stuart Penn, Visual Effects Supervisor, Framestore

Unlike his unruly predecessor, Smart Hulk is capable of graceful and delicate movements. “You wouldn’t have caught him picking up a pencil and poking the keyboards, but now he is,” states Penn. “These are precise actions, rather than Hulk smashing, grabbing a car and throwing it. It was a different level of animation. It was much more finessed.”

Having Smart Hulk interacting with the console was straightforward. “We found and filmed people with chunky hands to see the way all of the muscles and tendons moved and simulated that on top as well. We had him eating ice cream as well. We were trying to work out what sort of instrument does he use? Does he use a big or little spoon? His hands are disproportionately big to his mouth. It was tricky finding the right size spoon for him to hold in his fingers without it looking uncomfortable.

The entire hangar environment was built in CG along with the exterior view of the Avengers’ compound. 

“You wouldn’t have caught him picking up a pencil and poking the keyboards, but now he is. These are precise actions, rather than Hulk smashing, grabbing a car and throwing it. It was a different level of animation. It was much more finessed.”

—Stuart Penn, Visual Effects Supervisor, Framestore

The time-travel suits were a digital creation. 

“Our animators love to perform and filmed each other acting out things so they could then use that as reference beyond what we had,” adds Penn. “We got the designs of the costumes and built real copies of them. We dressed up the biggest and chunkiest in the office and filmed them to give us reference as to how the costume might move. Mark isn’t anywhere as heavy as Hulk and doesn’t move in the same way. His footsteps are nowhere near as big as Hulk, so it’s a definite interpretation of Mark’s body performance into something much bigger. Trying to keep it alive and weighty was definitely a challenge.”

Smart Hulk is slightly less green in Endgame, according to Penn. “There was a desire that we feel the human subsurface skin coming through underneath. We put a pinkish, subsurface glow underneath. We pushed that quite a lot, especially in the cheek area and around the ears, to give him more of a human, fleshy surface underneath the green that’s on top.”


Share this post with

Most Popular Stories

THE RETURN OF HAND-DRAWN AND STYLIZED EFFECTS ANIMATION
27 September 2022
Exclusives, Film
THE RETURN OF HAND-DRAWN AND STYLIZED EFFECTS ANIMATION
If you’ve noticed that a raft of animated films, shows and visual effects projects have been toying with more hand-crafted effects animation lately, you’re not alone.
LOOKING AT HIS DARK MATERIALS THROUGH THE AMBER SPYGLASS
03 October 2022
Exclusives, Film
LOOKING AT HIS DARK MATERIALS THROUGH THE AMBER SPYGLASS
Extensive creature work sparks fantasy series finale.
FRAMESTORE GOES TO WAR FOR THOR: LOVE AND THUNDER
13 September 2022
Exclusives, Film
FRAMESTORE GOES TO WAR FOR THOR: LOVE AND THUNDER
Thor: Love and Thunder, the Guardians of the Galaxy, the Indigarr battle, a crystal temple and giant goats.
SOUTH AND EAST ASIAN VFX STUDIOS MEET LOCAL AND GLOBAL DEMAND
03 October 2022
Exclusives, Film
SOUTH AND EAST ASIAN VFX STUDIOS MEET LOCAL AND GLOBAL DEMAND
Inside the South and East Asian VFX industry boom.
THERE IS HORROR IN THE CLOUDS IN NOPE
07 September 2022
Exclusives, Film
THERE IS HORROR IN THE CLOUDS IN NOPE
Being conventional when it comes to the horror genre is something that does not apply to filmmaker Jordan Peele
cialis online buy cialis