SONY PICTURES IMAGEWORKS JOINS FORCES WITH THE MITCHELLS VS. THE MACHINES

By TREVOR HOGG

Real paintings were studied, with the visual development team providing actual paint strokes that could be used.

Sony Pictures Imageworks continues to push the boundaries of animation with Netflix release of The Mitchells vs. the Machines, which revolves around a father and his college-bound daughter trying to connect with each other as a sentient AI system attempts to takeover the world. “Spider-Verse shook us up, got us to think differently, invent new tools, and tear down all of these CG principles,” states Sony Pictures Imageworks VFX Supervisor Mike Lasker. “The Mitchells was the same thing. We had to come up with new approaches for depth of field and lens flares as well as a new way to guide the eye of the audience in a more painterly fashion.” 2D sketches reflecting the state of mind of the protagonist voiced by Abbi Jacobson are mixed with the 3D animation. “Katie-Vision was Katie drawing on top of what was already animated,” Lasker adds. “It was mostly done by our Production Designer Lindsey Olivares and her team. The challenge with Katie-Vision is that we had to create a pipeline because they were giving us shot elements.”

Visual development art by Ian Worrel of the robot invasion.

As part of the illustrative animation style, a lot of effort went into emulating the look and feeling of the concept art in 3D. This piece of visual development was conceived by Jake Panian.

Michal Lasker, VFX Supervisor, Sony Pictures Imageworks

“We looked at how real paintings were done, and the visual development team sent us actual paint strokes that we used. A tool was built that used various types of projections on everything. The trick is that you don’t want it to look texture-mapped. As the characters move through the light, all of these watercolor strokes would organically build on top of each other to create shadows and light. That was one of the biggest tools that we used because it was over everything.”

Creatively, it was important not to hold back on ideas and experimentation. “When you’re creating brand-new looks the worse thing that you can do is inch your way there because you’re never going to get there,” states Lasker. “You need to go nuts, boldly try different things, and see what they like.” Whereas Spider-Man: Into the Spider-Verse had ink-inspired linework, The Mitchells vs. the Machines utilized a style that resembled a watercolor marker. “We needed to emulate the variation, breakup and the naturalism of the stroke; it would also come off of the surface sometimes and light would have to react to it.” The most expensive prop was the jacket worn by Rick Mitchell (Danny McBride). “Rick was our development testbed. He was the first character that we built. The jacket became indicative of the whole look.”

Understanding the nature of watercolors was an important aspect of the look development. “We looked at how real paintings were done, and the visual development team sent us actual paint strokes that we used,” remarks Lasker. “A tool was built that used various types of projections on everything. The trick is that you don’t want it to look texture-mapped. As the characters move through the light, all of these watercolor strokes would organically build on top of each other to create shadows and light. That was one of the biggest tools that we used because it was over everything.”

When everyone involved with the project bought into the look of The Mitchells vs. the Machines, the process became a lot smoother, says Lasker. “Once we had nailed down different frequencies and styles of brushstrokes that you pick for different scenarios, we stopped getting notes on that stuff.”

Complicating matters were the extremely aesthetic differences between the human and robot worlds. “All of these tools that we built had to be subtly scaled up and down,” explains Lasker. “If you look at linework on Rick and Linda Mitchell [Maya Rudolph] when they’re in PAL World, it’s much cleaner than when they were at the beginning.” A unique effect were the thrusters of the robots. “Under each boot, the effects team was able to create these polygonal simulations. You have the core of the thruster which was these triangles. Then they had these graphic outlines to frame them. On top of that you would get these polygonal vapor trails.” Explosions were given a painterly treatment. “We would create explosions with smoke and fire but first use painterly textures in the simulation. Then we would have 2D animators draw their effects on top of it. Once both were in the composite, we were able to mix them together and go with crazy colors if needed. It took a while to nail it.”

A mood board created by Lindsey Olivares of the Mitchells encountering the robot invasion.

Mood boards created by David R. Bleich explore the sleek and neon visual language of the robot world.

Left to right: Fred Armisen as “Deborahbot 5000” and Beck Bennett as “Eric,” Mike Rianda as “Aaron Mitchell,” Abbi Jacobson as “Katie Mitchell,” Danny McBride as “Rick Mitchell” and Maya Rudolph as “Linda Mitchell.”

A homage to Gremlins is the attack of the Furbies that takes place in a deserted shopping mall.

Geometric shapes and sleek designs in the robot world contrast with the homemade and chaotic human aesthetic.

“The Stealthbots were created by PAL, so they’re not going move like a person. Animation created this tool to slice up the geometry so that the robot could come apart in the most optimized way and find its next pose in the most optimized way. When you slice up the geometry you’re changing UVs and the topology, and we also needed the inside of these faces to glow. It went against everything that we would typically do.”

—Mike Lasker, VFX Supervisor, Sony Pictures Imageworks

A signature action sequence known as the Furby Attack resembles something out of Gremlins. “Gremlins came up a lot especially when you see them all lined up on the railings in the mall,” states Lasker. “One of my favorite effects is the giant Furby that has a Godzilla-esque beam coming out of its mouth. We landed on a cool middle ground where you could tell how hot the rubble was. Every object in the mall had to be brought to life and have its own personality.” Another aesthetic were the home videos. “[Director] Michael Rianda and [co-director] Jeff Rowe wanted a lighting scheme with an illustrated look. When you’re in the car the outside is blown out and the inside is dark because the exposure is not perfect, because they wanted the lighting to feel found. A lot of the home-video look is this raw lighting from this old camera, but we’re also having to make everything illustrated. We pushed the saturation, and played with chromatic aberrations and noise breakup.”

The beginning of the action family comedy has a camera style that resembles a handheld student project, and by the end the visuals are slick and smooth. “So much of these styles are related to the type of lensing,” notes Lasker. “If you want something to look graphic, the camera has to play to that. If you’re fighting 3D too much it’s going to be harder to make things look illustrated.” Continuity remains a challenge as hundreds of shots are being produced at the same time. “The characters had different levels of being dishevelled. Their hair gets slowly more messed up as they get more into act three. The car gets more destroyed over the course of the movie. We had to track it all.”

Breaking the animation pipeline were the Stealthbots. “Alan Hawkins [Head of Character Animation] had this great idea,” remarks Lasker. “If artificial intelligence creates something, what does it look like? If you look it up, there are weird, scary- looking things that a robot has designed that a human never would because it has been designed in the most optimal way. It has weird curves to it. The Stealthbots were created by PAL, so they’re not going move like a person.

Animation created this tool to slice up the geometry so that the robot could come apart in the most optimized way and find its next pose in the most optimized way. When you slice up the geometry you’re changing UVs and the topology, and we also needed the inside of these faces to glow. It went against everything that we would typically do.”

“Spider-Verse shook us up, got us to think differently, invent new tools, and tear down all of these CG principles. The Mitchells was the same thing. We had to come up with new approaches for depth of field and lens flares as well as a new way to guide the eye of the audience in a more painterly fashion.”

—Mike Lasker, VFX Supervisor, Sony Pictures Imageworks

Breaking the animation pipeline at Sony Pictures Imageworks were the Stealthbots, which were treated as if they were created by AI.

A handheld approach was adopted for the scenes in the human world involving the Mitchells as if Katie Mitchell (Abbi Jacobson) was putting together a student movie.

Polygonal vapor trails were created by the effects team for the robots.

The Mitchells vs. the Machines has a linework style that resembles a watercolor marker.

A variety of facial expressions were developed for Monchi.

“You see tons of trees and grass, so you need to find a way to make them look simple and artistic. On Spider-Verse, we developed this tool for the look of Gwen’s World because she had this artistic look going on. When I came onto Mitchells, I felt it was the tool that we needed to use. We started working on it again and created a way to take our raw render of grass and simplify into brushstrokes. … We had to build our trees in a certain way that they had enough leaf detail but also had geometry inside of it to create a simple, yet busy underlying coat of detail that we would then simplify. In order to simple things artistically you need detail underneath it. How we did the trees and grass was one of the most successful stylistic treatments that we did.”

—Mike Lasker, VFX Supervisor, Sony Pictures Imageworks

A favorite tool was the ability to turn the settings into painterly landscapes. “So much of the frame ends up being vegetation,” observes Lasker. “You see tons of trees and grass, so you need to find a way to make them look simple and artistic. On Spider-Verse, we developed this tool for the look of Gwen’s World because she had this artistic look going on. When I came onto Mitchells, I felt it was the tool that we needed to use. We started working on it again and created a way to take our raw render of grass and simplify into brushstrokes. Basically, no one is going to paint every blade of grass in a field. You’re going to have these nice swabs of color. Then we had to figure out how to do trees. We had to build our trees in a certain way that they had enough leaf detail but also had geometry inside of it to create a simple, yet busy underlying coat of detail that we would then simplify. In order to simple things artistically you need detail underneath it. How we did the trees and grass was one of the most successful stylistic treatments that we did.”

“Rick and Linda in the kitchen talking about Katie was the scene where we landed on the painterly style. That’s one of my favorites from a look standpoint,” reveals Lasker. “When Katie is in front of that window at the Dino Stop, the explosion happens, she is falling back in slow-motion, you see the Slurpee, she lands and the robots show up. We pushed the style of the movie more in that sequence than anywhere else. They were graphic, saturated, and had this light shining on them. Then when the robots capture the humans and Rick is flying through the air, the clouds have a beautiful, saturated style to them. The lighting schemes of those two sequences were my favorites. It had our style, action, color and great effects. It was everything that was great about the film.”

Katie-Vision occurs when Katie Mitchell (Abbi Jacobson) doodles on the screen to illustrate her inner emotions.

Driving the narrative is the emotional reconciliation that takes place between Katie Mitchell (Abbi Jacobson) and her father Rick Mitchell (Danny McBride).

Explosions were given a painterly treatment.

Share this post with