DIVERSITY REIGNS SUPREME IN SPIDER-MAN: ACROSS THE SPIDER-VERSE

By TREVOR HOGG

Images courtesy of Sony Pictures Animation.

Much like The Matrix, which caused a ripple through the film industry because of innovative storytelling, Spider-Man: Into the Spider-Verse had the same impact, with its creative influence going beyond its Oscar win for Best Animated Feature to inspire several imitations. Upping the ante is the first of two proposed sequels, Spider-Man: Across the Spider-Verse, by showcasing not only the world inhabited by a slightly older Miles Morales but those serving as the homes of Spider-Gwen, Spider-Man 2099 and Spider-Man: India. And let’s not forget that the cast of Spider-People has been multiplied by 10, and Miles finds himself clashing with them on how to handle the portal-empowered villain known as the Spot.

To handle the expanding multimedia aesthetic of the franchise, Sony Pictures Imageworks created a new department called Look of Picture. “On this one, there are so many technical hurdles and different looks that I’ve spent 80% of my time developing tools,” states Bret St. Clair, Senior Look Development and Lighting TD at Sony Pictures Imageworks. “We saw a lot of art that depicted paint being thrown around in-frame in ways where strokes aren’t specifically connected to a character or to anything. They’re floating in space. The problem for us is we try to approach everything rendering in a traditional way. Then we try to have all of our tools handle the rest in compositing. When it comes to positioning brushes in 3D, we needed to be able to reconstruct where those positions are, even when they don’t lie specifically on the surface that you’re applying the brush to, and the rendering tools, especially in 2D, don’t give you that information for free.”

Watercolor was the media of choice for Spider-Gwen’s world. “A lot of the look early on was inspired by the Jason Latour art, and everything was running vertically in that world,” St. Clair remarks. “As you start to move cameras around and we started to get through shots, it became clear we can’t have everything running vertically. It’s a balancing act because if you’re painting brushed volumes in a scene and the character starts to move, the brush volumes themselves get your attention as opposed to the thing you’re suppose to be looking at, or they give you the impression that they’re something else other than a volume. There is a lot of things that you can do in 2D that work because it’s a still frame, but as cameras move, it falls apart. Over the time we’ve been working on Gwen’s World, the tools of had to evolve so that the brushes automatically understand the surfaces that they’re painting onto, and painted in the right directions, or we have the ability to add hand-drawn strokes to something interactively.”

“There are so many technical hurdles and different looks that I’ve spent 80% of my time developing tools. We saw a lot of art that depicted paint being thrown around in-frame in ways where strokes aren’t specifically connected to a character or to anything. They’re floating in space.”

—Bret St. Clair, Senior Look Development and Lighting, Sony Pictures Imageworks

Linework is a major component to the look of Spider-Man: Across the Spider-Verse, with the tool development responsibility given to Pav Grochola, FX Supervisor on the film and Lead VFX Artist at Sony Pictures Imageworks. “When you’re creating linework, you’re starting off with a 3D model, and we try to make it look as hand-drawn as possible, which means that you’re trying to emulate the artistic process,” Grochola explains. “What an artist would do in the case of Medieval Vulture is Leonardo Da Vinci as much as you can. But all of that stuff is camera dependent, so you can’t be 100% procedural because 100% procedural looks like toon shading. We actually created our linework inside of Houdini, and it was driven by artists. Linework itself was in curves. We had a dedicated team of artists looking after the linework just for Medieval Vulture. The way you make something has a big impact on how it looks, so as much as possible we try to create procedural linework in Houdini, but also have artists draw linework on keyframes. Then that linework would be interpolated by our systems. It was a combination of hand-drawn and procedural. That’s the big secret with this stuff.”

For the Spot, Sony Picture Imageworks worked with Slovakian creative software company Escape Motions. “In terms of the technique, we did something interesting in this film in that we partnered with Escape Motions, which creates this painterly software called Rebelle that has an inbuilt watercolor solver, and you can do cool stuff like wet the canvas,” Grochola remarks. “You can put ink [on the wet canvas] and the ink spreads into the paper. We always tried to simulate the natural organic detail of painting in real life. Right at the beginning, we were working with that tool, and one of our biggest goals was to try to make the movie look as hand-drawn and handmade as possible; that tool to us was the perfect thing for experimenting with natural media. Not only can Rebelle do watercolor but oil paint and charcoal in such a convincingly way that would be hard for us to develop from scratch. We merged that software into our software and created this cool 3D and 2D combination where the Spot is made of paint, ink, and moves around leaving ink droplets; he is himself constantly redrawn with natural media.”

Medieval Vulture is made out of sepia tone paper. “We have him as translucent like paper so he scatters light through,” remarks Mike Lasker, VFX Supervisor at Sony Pictures Imageworks. “His character is probably one of the most complicated characters we have ever made because he is this Leonardo Da Vinci Medieval-style Vulture character with tons of pullies, feathers, and he’s got all of this stuff going on plus all of his linework. A lot of linework we do hand-drawn also. We mix actual 2D-drawn lines in with a lot of our stuff to give it that last piece of hand-drawn quality. But you need materials. What was a challenge on the first one as on this one is, what does a Da Vinci metal look like versus a Syd Mead? We do our traditional look development to a point and then we hand it off to our Look of Picture team where we do a style on top of that. I was just looking at an environment yesterday, and it’s in a world of pistons and gears and you’re in an entire environment of metal. Our look development looked realistic, but I tried to get the team to paint enough texture paint in there that is in the style so when we apply the Look of Picture tool it all works cohesively.”

Simulations have to take on the characteristics of the world in which they occurred. “In just the principles of film work and photography, we have to come up with how do we do depth of field in a painterly way versus a more architectural way versus how we do it in Miles World versus in India,” Lasker notes. “You have to experiment and be constantly ready to fail over and over again or at least try a lot of different things. One of the things we learned on the first one is you can’t inch your way to the finish line. You have to overdo it, and then figure out, what do we like from that, what is working and what is not? What does a lens flare look like in Spider-Man 2099 world versus Spider-Man: India? Every aspect you have to reinvent. The first film was really just one look.” Explosions have to be stylized and believable. “A lot of it is constantly playing with exposure, blowing out the lens, and how the light from the explosion affects the world around it,” comments Lasker. “If the explosion is in Gwen’s World, you want to feel the heat of the fire on the surfaces, and the shadows being cast are brushed like a painting, or the warmth on the surfaces needs to be brushed. How do those brushes react differently to what has already been brushed in the environment? You’re not only creating a painting, you’re having to paint the affects of the effect on the environment.”

“The way you make something has a big impact on how it looks, so as much as possible we try to create procedural linework in Houdini, but also have artists draw linework on keyframes. Then that linework would be interpolated by our systems. It was a combination of hand-drawn and procedural. That’s the big secret with this stuff.”

—Pav Grochola, FX Supervisor/Lead VFX Artist, Sony Pictures Imageworks

Spider-Punk evolved in animation. “We had to figure out all of the ways to handle his layers, and when you have a character who is meant to look like a magazine cut-out in a fleshed-out world that already has all kinds of crazy art directions in it, it’s hard to make it all cohesive,” observes Alan Hawkins, Head of Character Animation at Sony Pictures Imageworks. “But we found a formula once we got into the shots, like offsetting the frame rates of certain aspects of his body from other parts of his body. There was speculation that on the first one that there was meaning behind frame rates, but that was not the case. His jacket sometimes is a different frame rate than his body, and it’s to give him a patchwork collage feeling, and there are some slightly different layers to him being pasted together, which we did a lot of tests on; he is like 3s and 4s sometimes whereas everyone else is 2s.” The voice acting of Daniel Kaluuya was surprising. “It had such a swagger to his read that made the character so clear to us. Spider-Punk is probably one of my favorites now because he is consistent in his presentation to everyone else, except for a few key moments where you get to see his real truth, and that’s an amazing thing that we got to work with.”

“In the upgrade of Miles from a young teen to an older teen, his growing up a little was one of those where when the new shapes came into play, we handled it wrong; he no longer looked like himself,” Hawkins reveals. “Miles has a particular cheekbone structure and eye-corner shaping; his silhouette from the side and the way his mouth in the first movie had a bit of an overbite, and as they aged him up a little bit, you want to give someone a more pronounced jawline. That stuff did affect the way Miles looked, and he became a little unfamiliar. We had to make adjustments to how the rig would behave and how we handled the posing to make him look like a grownup version of that same Miles everybody loves.”

A massive focus was placed upon cinematography, composition and camera language. “That’s one of the things that we would say to every animator when they would join the show: you have to learn that language about overs, French overs, two shots, what lenses mean, which lenses to use in certain circumstances. Every animator was basically an honorary DP on this film because that was half of what we were handling.” Hawkins would love to do a reference cut for the whole movie. “I would like to point out how much the animators’ soul goes into these performances. For every scene where you see someone crying up there, that animator probably got to that place and filmed themselves doing it and re-referenced that stuff. It’s so complex and adult in a great way that we don’t often get in animation.”

Masks were incorporated into texture paint workflow. “For the texture painter, it’s color, bump, roughness and maybe a metallic,” observes texture painter Nicolle Cornute-Sutton. “Those are the traditional PBR types. On Spider-Verse, the laws of physics don’t apply, so physically-based rendering means nothing! Metal doesn’t shine like metal in our world. You have to think of things from a 2D perspective. For example, there is a dinosaur in our movie, and usually we would displace the scales to sell this bumpy, scaly reptilian creature, but you can’t do that in the Spider-Verse, so it’s almost working in a drop shadow and tracing things and giving it a more graphic read.” Templates were created. “The art department wanted it more stylized, and the only way you can do that is by hand. As our team grew, we had to make sure that we laid down foundations in Mari for new painters. We had to develop a lot of templates using node graphs so people could pick up a template and at least have a starting place so no one had to figure out for themselves, what is Miles’ World? Then we had to have a robust library of textures to show people that this is the target. That was new to me having to start from scratch.”

A complex battle that takes place in the Guggenheim Museum. “It’s in Gwen’s World, but the Guggenheim is actually filled with photorealistic artwork created by Jeff Koons,” Cornute-Sutton states. “It was one of the only times in the movie where we actually did go into Substance 3D Painter. We had photorealism, an architecturally significant building that we had to recreate, but in Gwen’s World style, and there is a helicopter bursting through the skylight in the ceiling. Vulture is in this sequence, and he’s from the Leonardo Da Vinci Verse, so he’s in a completely different sepia tone style. Then Spider-Man 2099 comes in with this digital suit that comes on in bits and bytes; his righthand gal Lyla pops in and she’s a hologram. Jessica Drew hops in on her motorcycle. And there’s Gwen and her father. They’re all in their styles.”

Diversity is paramount in the Spider-Verse. Observes Cornute-Sutton, “I felt like this was the first time I got to paint people who look and sounded like me and my family. That was incredibly exciting and thrilling thing. The audience is going to see that and feel that too. It’s not just about race or gender. I hope that this is just the beginning of many films where there’s a cognizant realization on the part of the filmmakers that we all want to see ourselves in CG features. I cannot help but say kudos to Sony Pictures Animation for taking a chance on making a movie that speaks to a broader range of audience than has been done before.”