VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


April 06
2022

ISSUE

Spring 2022

THE MATRIX RESURRECTIONS: WELCOME TO THE REAL WORLD … AGAIN

By KEVIN H. MARTIN 

Images courtesy of Warner Bros. and DNEG. 

DNEG referenced Geof Darrow’s original sketches from the first film in helping to recreate and enhance various environments being revisited in The Matrix Resurrections. These locales include the gel-filled pod containing Neo’s living body.

DNEG referenced Geof Darrow’s original sketches from the first film in helping to recreate and enhance various environments being revisited in The Matrix Resurrections. These locales include the gel-filled pod containing Neo’s living body. 

If the original Star Wars stood as the iconic representation of visual effects innovation for the last decades of the 20th Century, then 1999’s The Matrix surely had taken up that standard in the intervening years. Two sequels – also written and directed by the Wachowskis – may have diluted some of its impact, but the sheer excitement of the first film’s innovative bullet-time scenes raised the bar on VFX in a no-going-back-from-here way that was much aped, but rarely equaled. 

Working alone this time, writer-director Lana Wachowski’s return to the franchise with The Matrix Resurrections effectively subverts expectations of both fans and casual moviegoers, choosing to focus on character over flash while still managing to expand on both ‘real’ and virtual realms. For visual effects, this meant revisiting some creatures and environments from the original trilogy, but endowing them with greater verisimilitude; in equal parts, this called for creating new characters from a combination of on-set motion capture and CGI. 

As a veteran of the original two Matrix sequels, Visual Effects Supervisor Dan Glass offered an informed perspective on the new film’s place in the Matrix pantheon. “There was some trepidation,” he admits, “in terms of meeting audience expectations. But the goal was always to treat the film as its own entity; while it was written as a continuation, Resurrections also presented its own solid story. Visually and aesthetically, there was a deliberately chosen departure in its approach, something quite apart from its predecessors.” 

Part of that visual distinction came from Wachowski’s intent to shoot more in the real world and rely less on static compositions that harken back to graphic novels. “We used previs on a limited basis, mainly for the fully CG shots, so we understood how they laid out and how a camera might move through them,” states Glass. “There were also previs studies to figure out logistics in bigger sequences, but the main stylistic difference comes from being out on location rather than being studio-based, and Lana wanting to have tremendous freedom with the camera. So this one has a lot more fluidity; not necessarily a documentary look, but with more organic aspects of shooting out in the world. It was justified from a story standpoint because we’re depicting a different and more advanced version of the Matrix, where they’ve learned lessons about what constitutes reality. So our approach, to incorporate as much reality into things as possible, permitted a lot of impressive visuals, like seeing our two lead actors jump off the 40th story of a skyscraper.” 

The Dojo sequence was a callback to the Neo/Morpheus fight in the first film, and stylistically differed from the look of the real world and the matrix world.

The Dojo sequence was a callback to the Neo/Morpheus fight in the first film, and stylistically differed from the look of the real world and the matrix world. 

“[T]he main stylistic difference comes from being out on location rather than being studio-based… So this one has a lot more fluidity; not necessarily a documentary look, but with more organic aspects of shooting out in the world. It was justified from a story standpoint because we’re depicting a different and more advanced version of the Matrix, where they’ve learned lessons about what constitutes reality. So our approach, to incorporate as much reality into things as possible, permitted a lot of impressive visuals, like seeing our two lead actors jump off the 40th story of a skyscraper.” 

—Dan Glass, Senior Visual Effects Supervisor, DNEG 

Instead of simply revisiting bullet-time, Resurrections instead features scenes that essentially show the process through the other end of the scope, with time dilation being used against Neo. “To portray these moments, we built scenes using multiple frame rates for various characters and events,” Glass explains. “This involved referencing underwater photography to get ideas for how things might look and feel as Neo tries desperately to react to events transpiring faster than he can effectively defend against. We used lots of complex layering of CG over original photography, and in a few scenes employed a very subtle use of fluid dynamics, but it wasn’t about creating a big visual moment, and instead really focused on the emotional content and the acting.” 

Even with the focus often being on the intimate, there were still action set pieces, which range from a bullet-train battle to a chase through the streets of San Francisco. “Neo and Trinity race a motorcycle while trying to evade menacing throngs trying to stop them,” says Glass. “In terms of complexity, it was immense. We shot on location in the city with the principals on a real bike – rigged to a platform. Digitally, we went in and added even more figures to the thousands in the crowd while removing rigging.” 

DNEG added environmental effects to provide nuances such as rippling water and falling leaves.

DNEG added environmental effects to provide nuances such as rippling water and falling leaves.

DNEG added environmental effects to provide nuances such as rippling water and falling leaves. 

Numerous hoses plugged into Neo’s body were created in CG, and had to be tracked to actor Keanu Reeves’ moves and to his underlying musculature. The hoses also had to appear to interact with the red gel and various microbot creatures maintaining the pod.

Numerous hoses plugged into Neo’s body were created in CG, and had to be tracked to actor Keanu Reeves’ moves and to his underlying musculature. The hoses also had to appear to interact with the red gel and various microbot creatures maintaining the pod. 

Pre-production artwork suggested a path for realizing the environment surrounding the pods of Neo and Trinity, with production building a 10-meter-high set piece including a practical pod.

Pre-production artwork suggested a path for realizing the environment surrounding the pods of Neo and Trinity, with production building a 10-meter-high set piece including a practical pod. 

New visual sensibilities – rendered with latticeworks and curved forms – contrast with the previously established brutalist look, suggesting how the faction of Machines working and living with the humans have created a new hybrid aesthetic.

New visual sensibilities – rendered with latticeworks and curved forms – contrast with the previously established brutalist look, suggesting how the faction of Machines working and living with the humans have created a new hybrid aesthetic. 

For the first time with their feature work, DNEG relied heavily throughout the scene on Epic’s Unreal engine for real-time rendering at 4K resolution.

For the first time with their feature work, DNEG relied heavily throughout the scene on Epic’s Unreal engine for real-time rendering at 4K resolution. 

The film utilized work from several vendors, including Framestore, Rise, One of Us, BUF, Turncoat Pictures and Studio C, with DNEG drawing a significant number of shots and sequences, including the Dojo sequence in which Neo and a new incarnation of Morpheus square off. Though DNEG created the CG dojo with Clarisse, DNEG Visual Effects Supervisor Huw Evans reports this was the first occasion when the company used real-time rendering with Epic Games’ Unreal engine for finals-quality renders at 4K resolution. “That was a big deal for us,” he avows. “Lana and Epic crafted the scene based on a place called Devil’s Bridge in Germany. After Epic built that out, we took it off their hands, adding extra detail such as rippling water and falling leaves, then starting lighting with it before running our CG cameras and match-moving the real cameras. 

“The big challenge at the time in running it all through Unreal was the limitations of version 4.25,” Evans elaborates. “That was missing a lot of features we’d normally use in dealing with imagery, like OCIO color support, that can help get us into comp with all the extra sprinkles. So after we got these beautiful images from Epic, we’d always put things into our pipeline to have the ability to tweak individual grades, such as varying bloom or emphasizing some aspect of a visual effect. We could have output straight from Unreal, but it would have been missing that 10% we always strive to achieve.” 

Glass notes that the decision to handle that sequence in Unreal was made in part because the scene represents a construct within the matrix. “There was an aesthetic choice for that to be different in look from the rest of the film, but internally consistent.” 

With the film’s new machine version of Morpheus, live on-set facial capture was used. “The real benefit here was getting all that interaction between the actors live so the eyelines were maintained naturally during the conversations,” Glass acknowledges. “The biggest R&D components on the project connected to the newer body-method capture, which used AI and machine learning to reconstruct from data these three-dimensional representations of actors. It didn’t require a CG head or do the modeling and lighting, because you get all that baked in from the original photography. That was something Lana really wanted because it went along with how she might want to pan rapidly from one direction to the other. The AI/machine learning tech was also used when two actresses are supposed to be occupying the same space – occupying the same avatar, if you will – so we recorded each of them, then used machine learning to superimpose one over another while they appear to be fully synchronized with all their actions.” 

Determining just how different this new Morpheus would appear and behave was an iterative process for DNEG. “While he looks human when inside the matrix, his digital look flows and changes as the situation demands,” says Evans. 

The resistance’s new belowground digs, Io, represented a departure from their older Zion enclave, suggesting a much more expansive abode, with factories and delineated residential blocs.

The resistance’s new belowground digs, Io, represented a departure from their older Zion enclave, suggesting a much more expansive abode, with factories and delineated residential blocs. 

“The big challenge at the time in running it all through Unreal was the limitations of version 4.25. That was missing a lot of features we’d normally use in dealing with imagery, like OCIO color support, that can help get us into comp with all the extra sprinkles. So after we got these beautiful images from Epic, we’d always put things into our pipeline to have the ability to tweak individual grades, such as varying bloom or emphasizing some aspect of a visual effect. We could have output straight from Unreal, but it would have been missing that 10% we always strive to achieve.” 

—Huw Evans, Visual Effects Supervisor, DNEG 

“The earliest attempts were very free-flowing, and I really quite liked the casually arrogant way he could let parts of himself flow ahead before snapping back into the human configuration. But in going down that route, we found that it was harder to relate to this major character when he was all over the place; your eye didn’t always know where to look when he was in such extreme motion. I decided that if he was looking at you, his face and arms would be mostly human, but you’d see this almost seagrass-like effect visible on his back, and a different level of flowing when his muscles were in use, like when he was climbing.” 

Separate muscle and skin passes were required before the flowing character effect went in. “There’s a lot of nuance to the performance that started with what the face camera got, in order to convey how he doesn’t have the fidelity of a real human,” states Evans. “When the effects team took over, they tried to proceduralize it as much as possible, but with so much custom work to make the character, there’s still a lot remaining.” 

DNEG was also responsible for aspects involving locales recognizable from the first film, including the pod containing Neo’s inert form as he lay in a pool of red gel, literally plugged into the Matrix through various hoses. “The look of that environment started with a beautiful piece of concept art that gave us the broad strokes,” recalls Evans. “Then we did countless designs on the specifics of the pods and the huge turbines around them. Production actually built a massive set piece for the live action that measured 10 meters high and included Neo’s pod – we reused that for Trinity’s, too – which contained all that practical goo. We extended the turbine and built a whole chamber around it in CG, along with the pit below.

For a Machine incarnation of Morpheus, the film utilized live on-set facial capture. AI and machine learning permitted reconstruction of the character from three-dimensional presences on set. Separate passes for skin and muscle helped finesse the look, but Morpheus’ ability to fluctuate his form went through many iterations before filmmakers settled on a subdued rippling to avoid distracting or confusing viewers.

For a Machine incarnation of Morpheus, the film utilized live on-set facial capture. AI and machine learning permitted reconstruction of the character from three-dimensional presences on set. Separate passes for skin and muscle helped finesse the look, but Morpheus’ ability to fluctuate his form went through many iterations before filmmakers settled on a subdued rippling to avoid distracting or confusing viewers.

For a Machine incarnation of Morpheus, the film utilized live on-set facial capture. AI and machine learning permitted reconstruction of the character from three-dimensional presences on set. Separate passes for skin and muscle helped finesse the look, but Morpheus’ ability to fluctuate his form went through many iterations before filmmakers settled on a subdued rippling to avoid distracting or confusing viewers. 

A flashback illustrating the war between two factions of Machines provided an opportunity to deliver exposition through a pair of expansive visual cuts.

A flashback illustrating the war between two factions of Machines provided an opportunity to deliver exposition through a pair of expansive visual cuts. 

The Machines’ armada reflected some souping-up from their earlier forms, while the opposition – operating in what were dubbed ‘squid tanks’ – used geometry made up of familiar detail bits from the trilogy.

The Machines’ armada reflected some souping-up from their earlier forms, while the opposition – operating in what were dubbed ‘squid tanks’ – used geometry made up of familiar detail bits from the trilogy. 

The faction of Machines working and living with the humans have created a new hybrid aesthetic.

The faction of Machines working and living with the humans have created a new hybrid aesthetic. 

These constructs were only slightly ‘upgraded’ – both to reflect the changes in the nature of the matrix as well as CG developments over the intervening years separating Resurrections from the original trilogy.

These constructs were only slightly ‘upgraded’ – both to reflect the changes in the nature of the matrix as well as CG developments over the intervening years separating Resurrections from the original trilogy. 

It was a fairly standard build, but to give the scene life and a tie-in back to the original film, we added a bunch of little microbot creatures for additional texture and detail. We referenced Geof Darrow’s original sketches from the first film [of the machine ecosystem] to keep these things looking familiar and appropriate, but we got to embellish things by giving them a sense of character when Neo unexpectedly wakes – they are surprised and get away really quickly!” 

Evans found one of the most challenging parts of the sequence to be a fairly invisible aspect. “The hoses that attach to their bodies were all digital,” he explains. “We tracked all those ports on Neo’s back individually – that meant not only matching the movement, but also understanding and reproducing the musculature beneath those ports, including how his skin moves and he reacts. These CG cables also had to interact with the practical goo that was in the pod and dripping off Neo, plus those microbot creatures as they splashed around in the goo; all of this amounted to incredibly detailed work that was often painful to get just right. None of this is really in-your-face stuff, but the invisible work is often at least as challenging. It made for just a ton of fine-detail balancing.” 

A throwaway line of dialog led to a short but memorable sequence featuring combat between two factions of warring machines. “The machine battle was originally just referred to in conversation, and then it was decided to do a single big shot of the action, which sounded exciting,” admits Evans. “What was even better was how it eventually became two massive shots. This started life as another piece of concept art that roughly blocked out what was going on. We used the armada ships from the original trilogy on one side of frame, updated with new detail, flying through the air, while on the other side we had a group in what we called squid tanks. They were a hybrid using familiar bits of geometry from the original, including elements from the harvester and sentinel, but with new weird dreadlocked bits that resembled sentinel legs. Lana was keen to not include the harvesters intact, because they are farming creatures, not built for firing lasers and doing battle. The Environments team did an amazing job with all the fire and smoke and debris, which was especially important in creating the spectacle, because since it was just two shots, we didn’t want to go crazy modeling all these forms and mainly built to camera whenever possible. 

Among the familiar ‘faces’ from earlier Matrix entries are the Machines’ attack Sentinels and Io’s hovercraft vessels, along with Harvesters used by the Machines to service fetus fields.

Among the familiar ‘faces’ from earlier Matrix entries are the Machines’ attack Sentinels and Io’s hovercraft vessels, along with Harvesters used by the Machines to service fetus fields. 

DNEG extended the turbine section and filled in the rest of the environment digitally.

DNEG extended the turbine section and filled in the rest of the environment digitally. 

Upgraded from earlier Matrix entries are the Machines’ attack Sentinels and Io’s hovercraft vessels, along with Harvesters used by the Machines to service fetus fields.

Upgraded from earlier Matrix entries are the Machines’ attack Sentinels and Io’s hovercraft vessels, along with Harvesters used by the Machines to service fetus fields. 

Belowground in Io.

Belowground in Io.

Environments, along with the matte painting team, also contributed heavily to Io, the new city that succeeds Zion as home to the human resistance, “If Zion represented a town, then Io is a mega-city – massive, with towering buildings, factories and delineated residential blocs within a sprawling cave environment,” remarks Evans. “We tried to generate stuff procedurally, when possible, but when there were specific story points, like being able to see people moving around in their residences and people here interact with these machines on their side, there had to be a great deal of detail work. Lana was very keen that this environment show how the combining of people and machines working together reflected a very different visual sensibility from that of Zion, which was very run-down and clearly just a product of human minds. With the help of this faction of machines, you go beyond just a Brutalist grouping of buildings into details featuring 3D-printed lattices and curved shapes made from unusual exotic materials.” 

To facilitate the film’s less-regimented approach to shooting, DNEG always used the production imagery as a reference. “Whenever we had a fully CG shot, especially one that went between two production shots, we made sure our CG cameras could match the moving and sometimes handheld look of the production shoot,” says Evans. That meant duplicating the exact kind of camera shake and bounce, but it was absolutely necessary to make everything live together in the cut. 

“Everybody loved what the first Matrix brought to cinema,” Evans concludes. “But if Lana had taken the easy way out and done the same exact thing over again 20 years later, it would have been treading on familiar territory that so many other projects have already leveraged off. This one defies a lot of conventional thinking about sequels, while having meta kinds of fun with the whole notion of sequels. That may not be what everybody expected it to be, but it is very different, and I found that aspect, along with getting to be part of the Matrix history, to be extremely worthwhile.” 


Share this post with

Most Popular Stories

CHANNELING THE VISUAL EFFECTS ON SET FOR THE WHEEL OF TIME SEASON 2
23 January 2024
Film
CHANNELING THE VISUAL EFFECTS ON SET FOR THE WHEEL OF TIME SEASON 2
In the case of the second season of The Wheel of Time, the on-set visual effects supervision was equally divided between Roni Rodrigues and Mike Stillwell.
2024 STATE OF THE VFX/ANIMATION INDUSTRY: FULL SPEED AHEAD
09 January 2024
Film
2024 STATE OF THE VFX/ANIMATION INDUSTRY: FULL SPEED AHEAD
Despite production lulls, innovations continue to forward the craft.
GAIA BUSSOLATI: CONNECTING HOLLYWOOD AND ITALIAN CINEMA – AND STAYING CURIOUS
09 January 2024
Film
GAIA BUSSOLATI: CONNECTING HOLLYWOOD AND ITALIAN CINEMA – AND STAYING CURIOUS
VFX Supervisor bridges Italian cinema, Hollywood blockbusters.
NAVIGATING LONDON UNDERWATER FOR THE END WE START FROM
05 March 2024
Film
NAVIGATING LONDON UNDERWATER FOR THE END WE START FROM
Mahalia Belo’s remarkable feature directorial debut The End We Start From follows a woman (Jodie Comer) and her newborn child as she embarks on a treacherous journey to find safe refuge after a devastating flood.
HOW TIME KEEPS SLIPPING AWAY IN LOKI SEASON 2
19 December 2023
Film
HOW TIME KEEPS SLIPPING AWAY IN LOKI SEASON 2
Created by Michael Waldron for Disney +, the second season of Loki follows notorious Marvel villain Loki (portrayed by Tom Hiddleston), Thor’s adopted brother and God of Mischief.