By KEVIN H. MARTIN
Images courtesy of Warner Bros. and DNEG.
The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.
Winner of three prestigious Folio Awards for excellence in publishing.
By KEVIN H. MARTIN
Images courtesy of Warner Bros. and DNEG.
If the original Star Wars stood as the iconic representation of visual effects innovation for the last decades of the 20th Century, then 1999’s The Matrix surely had taken up that standard in the intervening years. Two sequels – also written and directed by the Wachowskis – may have diluted some of its impact, but the sheer excitement of the first film’s innovative bullet-time scenes raised the bar on VFX in a no-going-back-from-here way that was much aped, but rarely equaled.
Working alone this time, writer-director Lana Wachowski’s return to the franchise with The Matrix Resurrections effectively subverts expectations of both fans and casual moviegoers, choosing to focus on character over flash while still managing to expand on both ‘real’ and virtual realms. For visual effects, this meant revisiting some creatures and environments from the original trilogy, but endowing them with greater verisimilitude; in equal parts, this called for creating new characters from a combination of on-set motion capture and CGI.
As a veteran of the original two Matrix sequels, Visual Effects Supervisor Dan Glass offered an informed perspective on the new film’s place in the Matrix pantheon. “There was some trepidation,” he admits, “in terms of meeting audience expectations. But the goal was always to treat the film as its own entity; while it was written as a continuation, Resurrections also presented its own solid story. Visually and aesthetically, there was a deliberately chosen departure in its approach, something quite apart from its predecessors.”
Part of that visual distinction came from Wachowski’s intent to shoot more in the real world and rely less on static compositions that harken back to graphic novels. “We used previs on a limited basis, mainly for the fully CG shots, so we understood how they laid out and how a camera might move through them,” states Glass. “There were also previs studies to figure out logistics in bigger sequences, but the main stylistic difference comes from being out on location rather than being studio-based, and Lana wanting to have tremendous freedom with the camera. So this one has a lot more fluidity; not necessarily a documentary look, but with more organic aspects of shooting out in the world. It was justified from a story standpoint because we’re depicting a different and more advanced version of the Matrix, where they’ve learned lessons about what constitutes reality. So our approach, to incorporate as much reality into things as possible, permitted a lot of impressive visuals, like seeing our two lead actors jump off the 40th story of a skyscraper.”
“[T]he main stylistic difference comes from being out on location rather than being studio-based… So this one has a lot more fluidity; not necessarily a documentary look, but with more organic aspects of shooting out in the world. It was justified from a story standpoint because we’re depicting a different and more advanced version of the Matrix, where they’ve learned lessons about what constitutes reality. So our approach, to incorporate as much reality into things as possible, permitted a lot of impressive visuals, like seeing our two lead actors jump off the 40th story of a skyscraper.”
—Dan Glass, Senior Visual Effects Supervisor, DNEG
Instead of simply revisiting bullet-time, Resurrections instead features scenes that essentially show the process through the other end of the scope, with time dilation being used against Neo. “To portray these moments, we built scenes using multiple frame rates for various characters and events,” Glass explains. “This involved referencing underwater photography to get ideas for how things might look and feel as Neo tries desperately to react to events transpiring faster than he can effectively defend against. We used lots of complex layering of CG over original photography, and in a few scenes employed a very subtle use of fluid dynamics, but it wasn’t about creating a big visual moment, and instead really focused on the emotional content and the acting.”
Even with the focus often being on the intimate, there were still action set pieces, which range from a bullet-train battle to a chase through the streets of San Francisco. “Neo and Trinity race a motorcycle while trying to evade menacing throngs trying to stop them,” says Glass. “In terms of complexity, it was immense. We shot on location in the city with the principals on a real bike – rigged to a platform. Digitally, we went in and added even more figures to the thousands in the crowd while removing rigging.”
The film utilized work from several vendors, including Framestore, Rise, One of Us, BUF, Turncoat Pictures and Studio C, with DNEG drawing a significant number of shots and sequences, including the Dojo sequence in which Neo and a new incarnation of Morpheus square off. Though DNEG created the CG dojo with Clarisse, DNEG Visual Effects Supervisor Huw Evans reports this was the first occasion when the company used real-time rendering with Epic Games’ Unreal engine for finals-quality renders at 4K resolution. “That was a big deal for us,” he avows. “Lana and Epic crafted the scene based on a place called Devil’s Bridge in Germany. After Epic built that out, we took it off their hands, adding extra detail such as rippling water and falling leaves, then starting lighting with it before running our CG cameras and match-moving the real cameras.
“The big challenge at the time in running it all through Unreal was the limitations of version 4.25,” Evans elaborates. “That was missing a lot of features we’d normally use in dealing with imagery, like OCIO color support, that can help get us into comp with all the extra sprinkles. So after we got these beautiful images from Epic, we’d always put things into our pipeline to have the ability to tweak individual grades, such as varying bloom or emphasizing some aspect of a visual effect. We could have output straight from Unreal, but it would have been missing that 10% we always strive to achieve.”
Glass notes that the decision to handle that sequence in Unreal was made in part because the scene represents a construct within the matrix. “There was an aesthetic choice for that to be different in look from the rest of the film, but internally consistent.”
With the film’s new machine version of Morpheus, live on-set facial capture was used. “The real benefit here was getting all that interaction between the actors live so the eyelines were maintained naturally during the conversations,” Glass acknowledges. “The biggest R&D components on the project connected to the newer body-method capture, which used AI and machine learning to reconstruct from data these three-dimensional representations of actors. It didn’t require a CG head or do the modeling and lighting, because you get all that baked in from the original photography. That was something Lana really wanted because it went along with how she might want to pan rapidly from one direction to the other. The AI/machine learning tech was also used when two actresses are supposed to be occupying the same space – occupying the same avatar, if you will – so we recorded each of them, then used machine learning to superimpose one over another while they appear to be fully synchronized with all their actions.”
Determining just how different this new Morpheus would appear and behave was an iterative process for DNEG. “While he looks human when inside the matrix, his digital look flows and changes as the situation demands,” says Evans.
“The big challenge at the time in running it all through Unreal was the limitations of version 4.25. That was missing a lot of features we’d normally use in dealing with imagery, like OCIO color support, that can help get us into comp with all the extra sprinkles. So after we got these beautiful images from Epic, we’d always put things into our pipeline to have the ability to tweak individual grades, such as varying bloom or emphasizing some aspect of a visual effect. We could have output straight from Unreal, but it would have been missing that 10% we always strive to achieve.”
—Huw Evans, Visual Effects Supervisor, DNEG
“The earliest attempts were very free-flowing, and I really quite liked the casually arrogant way he could let parts of himself flow ahead before snapping back into the human configuration. But in going down that route, we found that it was harder to relate to this major character when he was all over the place; your eye didn’t always know where to look when he was in such extreme motion. I decided that if he was looking at you, his face and arms would be mostly human, but you’d see this almost seagrass-like effect visible on his back, and a different level of flowing when his muscles were in use, like when he was climbing.”
Separate muscle and skin passes were required before the flowing character effect went in. “There’s a lot of nuance to the performance that started with what the face camera got, in order to convey how he doesn’t have the fidelity of a real human,” states Evans. “When the effects team took over, they tried to proceduralize it as much as possible, but with so much custom work to make the character, there’s still a lot remaining.”
DNEG was also responsible for aspects involving locales recognizable from the first film, including the pod containing Neo’s inert form as he lay in a pool of red gel, literally plugged into the Matrix through various hoses. “The look of that environment started with a beautiful piece of concept art that gave us the broad strokes,” recalls Evans. “Then we did countless designs on the specifics of the pods and the huge turbines around them. Production actually built a massive set piece for the live action that measured 10 meters high and included Neo’s pod – we reused that for Trinity’s, too – which contained all that practical goo. We extended the turbine and built a whole chamber around it in CG, along with the pit below.
It was a fairly standard build, but to give the scene life and a tie-in back to the original film, we added a bunch of little microbot creatures for additional texture and detail. We referenced Geof Darrow’s original sketches from the first film [of the machine ecosystem] to keep these things looking familiar and appropriate, but we got to embellish things by giving them a sense of character when Neo unexpectedly wakes – they are surprised and get away really quickly!”
Evans found one of the most challenging parts of the sequence to be a fairly invisible aspect. “The hoses that attach to their bodies were all digital,” he explains. “We tracked all those ports on Neo’s back individually – that meant not only matching the movement, but also understanding and reproducing the musculature beneath those ports, including how his skin moves and he reacts. These CG cables also had to interact with the practical goo that was in the pod and dripping off Neo, plus those microbot creatures as they splashed around in the goo; all of this amounted to incredibly detailed work that was often painful to get just right. None of this is really in-your-face stuff, but the invisible work is often at least as challenging. It made for just a ton of fine-detail balancing.”
A throwaway line of dialog led to a short but memorable sequence featuring combat between two factions of warring machines. “The machine battle was originally just referred to in conversation, and then it was decided to do a single big shot of the action, which sounded exciting,” admits Evans. “What was even better was how it eventually became two massive shots. This started life as another piece of concept art that roughly blocked out what was going on. We used the armada ships from the original trilogy on one side of frame, updated with new detail, flying through the air, while on the other side we had a group in what we called squid tanks. They were a hybrid using familiar bits of geometry from the original, including elements from the harvester and sentinel, but with new weird dreadlocked bits that resembled sentinel legs. Lana was keen to not include the harvesters intact, because they are farming creatures, not built for firing lasers and doing battle. The Environments team did an amazing job with all the fire and smoke and debris, which was especially important in creating the spectacle, because since it was just two shots, we didn’t want to go crazy modeling all these forms and mainly built to camera whenever possible.
Environments, along with the matte painting team, also contributed heavily to Io, the new city that succeeds Zion as home to the human resistance, “If Zion represented a town, then Io is a mega-city – massive, with towering buildings, factories and delineated residential blocs within a sprawling cave environment,” remarks Evans. “We tried to generate stuff procedurally, when possible, but when there were specific story points, like being able to see people moving around in their residences and people here interact with these machines on their side, there had to be a great deal of detail work. Lana was very keen that this environment show how the combining of people and machines working together reflected a very different visual sensibility from that of Zion, which was very run-down and clearly just a product of human minds. With the help of this faction of machines, you go beyond just a Brutalist grouping of buildings into details featuring 3D-printed lattices and curved shapes made from unusual exotic materials.”
To facilitate the film’s less-regimented approach to shooting, DNEG always used the production imagery as a reference. “Whenever we had a fully CG shot, especially one that went between two production shots, we made sure our CG cameras could match the moving and sometimes handheld look of the production shoot,” says Evans. That meant duplicating the exact kind of camera shake and bounce, but it was absolutely necessary to make everything live together in the cut.
“Everybody loved what the first Matrix brought to cinema,” Evans concludes. “But if Lana had taken the easy way out and done the same exact thing over again 20 years later, it would have been treading on familiar territory that so many other projects have already leveraged off. This one defies a lot of conventional thinking about sequels, while having meta kinds of fun with the whole notion of sequels. That may not be what everybody expected it to be, but it is very different, and I found that aspect, along with getting to be part of the Matrix history, to be extremely worthwhile.”