FuseFX Supplies UPLOAD with a Dynamic Digital Afterlife

By KEVIN H. MARTIN

Upload postulates a future in which concerns over quality of life carry quite literally into beyond corporeal existence. In 2033, humans possessing sufficient means can ‘upload’ their consciousness prior to death into a synthetic afterlife. Even less affluent people for whom the price is not right can avail themselves of basic-cable versions – though these low-rent alternatives seem markedly more like hell than heaven.

After dying as a result of an auto accident – a suspicious event in itself, almost unheard of in a time of utterly safe, self-driving vehicles – Nathan (Robbie Amell) returns to consciousness to find himself in Lakeview, one of the trendier spots for the newly dead. He is able to change the weather conditions outside his luxury hotel by just rotating a dial on what looks like an ordinary thermostat. Suffering from what turns out to be very selective memory loss, Nathan ultimately proves that in death he can be more of a man than he ever was while alive and walking the planet, as his real-world Lakeview CSR Nora (Andy Allo) finds during their increasingly intimate virtual interactions.

Marshall Richard Krasser, VFX Supervisor, FuseFX

Series creator Greg Daniels (Parks and Recreation, Space Force and the American version of The Office) had first come up with the idea for this series back in the 1990s when he was writing for Saturday Night Live. He chose VFX Supervisor Marshall Richard Krasser and the Emmy-winning FuseFX to handle the visual effects. Krasser, with a substantial work history in compositing, transitioned into supervision during his lengthy tenure at ILM, working on features that included Pearl Harbor, Harry Potter, Star Trek and Iron Man 2. “Even though I came up from a 2D and compositing background, there were still shows when I would light and do other kinds of technical work,” Krasser reports. “I came to understand roto and cleanup, plus at ILM we had practical work continuing for quite awhile into the prequels, so I’d be supervising when we went outside to blow things up, generating elements for the final.”

When he worked on Steven Spielberg’s War of the Worlds, ILM turned the whole 700-shot show around in three months. “There was not a lot of room for wishy-washy decisions, so we just buckled down and put the team together, splitting it into separate units headed by Dennis Muren and Pablo Helman,” recalls Krasser. “I comp-suped both of them, switching back and forth between their dailies to ensure consistency for both units. It wasn’t a matter of setting a tone for them, but more a matter of informing one group, ‘hey, we’re already doing that in this way over here, so you guys should talk to make sure we’re all going in the same direction. So that kind of prepared me for supervising.” Krasser also supervised a number of TV series while at Psyop and CoSA VFX, including The Man in the High Castle, The X-Files revival, Waco and Life Sentence.

“[Series creator Greg Daniels] told us he knew our work and liked it, but that the most important aspect was how we had done visual effects for comedy and knew how to make it work for those very different needs. I had served as the compositing supervisor on Galaxy Quest and remembered that even back then, it was a delicate matter to make things funny without having them get laughed at. And there’s also the issue of modulating humor so it doesn’t come at the expense of storytelling.”

—Marshall Richard Krasser, Visual Effects Supervisor

Before his fatal auto accident, Nathan (Robbie Amell) checks out at the market, processed by a robot cashier. The first animation tests for the robotic arm proved too funny, undercutting the story, and were subsequently scaled down. (Photo courtesy of Marshall Richard Krasser)

Street background was a combination of location plate elements and CG auto. Except for a mockup used for interiors that was shot greenscreen, all of the driving scenes were full CG scene simulations, sweetened only with real hills behind the roadway and vehicles. Noise was added to give the vehicles a bit of bump and shake. (Photo courtesy of Marshall Richard Krasser)

“Upload had a scene with a cashier who is basically just a robotic arm done in CG, Krasser elaborates. “Our animator, Adam Chaput, had a lot of fun with it at first, creating a hilarious character who had Greg laughing a lot when we showed them. But then he told us, ‘Guys, I hate to do this to you, but it’s too funny. The robot is upstaging my actors. Can you dial it back?’ … So this was in keeping with my usual preference for taking stuff too far before reigning back.”

—Marshall Richard Krasser, Visual Effects Supervisor

Series creator Greg Daniels acts out a move for actress Andy Allo (Nora) to emulate while airborne. (Photo courtesy of Marshall Richard Krasser)

“When we had our initial meeting with Greg Daniels,” Krasser recalls, “he told us he knew our work and liked it, but that the most important aspect was how we had done visual effects for comedy and knew how to make it work for those very different needs. I had served as the compositing supervisor on Galaxy Quest and remembered that even back then, it was a delicate matter to make things funny without having them get laughed at. And there’s also the issue of modulating humor so it doesn’t come at the expense of storytelling.

“Upload had a scene with a cashier who is basically just a robotic arm done in CG, Krasser elaborates. “Our animator, Adam Chaput, had a lot of fun with it at first, creating a hilarious character who had Greg laughing a lot when we showed them. But then he told us, ‘Guys, I hate to do this to you, but it’s too funny. The robot is upstaging my actors. Can you dial it back?’And I totally got that. But it did prove that going with a sense of character was the right direction, because if we’d just handled it as a robot without that performance aspect, it would have felt a little off from the rest of the Upload universe. So this was in keeping with my usual preference for taking stuff too far before reigning back.”

The dialog between Daniels and VFX was extensive and informative. “Greg had refined the original idea a number of times over the years, and was able to carry the whole thing in his head,” marvels Krasser. “I prefer it when a showrunner knows what he wants, and Greg was all that. But I also like being a collaborator rather than simply providing a service, so I made some suggestions that took it away from what was first planned, and Greg was supportive. I might look at what we were going to be shooting and suggest that a particular effect should be achieved in-camera or possibly through a makeup effect. There are a lot of potential solutions, so you shouldn’t limit your creative thinking just because the default mindset is to do it in post. And at the very least, what we shoot on set can serve as reference for VFX, so the shot will have a certain grounding to it. I’ve been trained to go out and find real-world references rather than reference another movie for what happens in real life.”

When FuseFX came onboard Upload, the pilot episode had already been shot and largely completed. “There were some early VFX in place,” acknowledges Krasser, but they weren’t finals. We re-did pretty much all of it, but that work served us well as a kind of high-level animatic. So we’d still have discussion about how to execute a given visual, whether this should be handled in 2D or 3D. You have to make smart choices, not just for the budget but also for maximizing your in-place resources.

“I’ve had occasion in the past to tell somebody working in 3D that they only needed to detail or texture the front side of the model,” he expounds, “because we weren’t ever going to see the back part. Everybody needs to imagine it from a camera perspective; it is just like an old Hollywood backlot, where things look great from the street side, but you walk around the corner and the man behind the curtain is revealed: you realize it is all false-front cutouts. Then there was one movie where they had us build all the detail down to seeing threads on the screws. I’d have to point out to people that the model was turning out to be so heavy that you couldn’t open up the scene file. Don’t invest resources in things nobody will ever see.”

The work largely broke up in two different styles to differentiate between the real world and the digital afterlife sims. “The scenes in Lakeview are often static or with more constrained camera moves,” says Krasser. “Life out in the world of Earth 2033 was intended to look a bit grittier, while featuring more of a mobile camera.” The dynamics in reality are very much on display in many driving scenes, which became a showcase for standalone VFX. “The exterior driving shots are pretty much fully synthetic, except for the distant hills,” Krasser relates. “Those nice brown hills were based on stills of mine taken right before I went to work on the series, when I took a trip to Joshua Tree.”

CG Supervisor Rav Brar oversaw development on the oft-seen freeways, building out a section and replicating that as needed. “We rendered out various trees and foliage using SpeedTree,” states Krasser. “Production had built a car mockup and the original VFX had followed that, so we had a template for the various vehicles. But one problem with the idea of all these self-driving cars is that they behave perfectly – which works against the idea that you’re seeing a real environment, where the vehicles would be changing speeds from the heavy foot on the pedal or somebody braking early. We wound up treating it like cars in a ride at Disneyland, which limited us in terms of where we could go with the flow – until or unless we get a drunk car owing to bad octane. We did try to offset that artificial/video game feel ever-so-slightly by introducing a bit of noise to give the vehicles some road bounce. That together with some camera vibration as well helped take the curse off it.”

Nathan attends his own virtual funeral, with real-live mourners seen opposite him. Bluescreen was used here to avoid extraction issues with the room’s green tones, but greenscreen also saw action throughout. (Photo courtesy of Marshall Richard Krasser)

A cyberspace black market requiring later VFX was another bluescreen setup. (Photo courtesy of Marshall Richard Krasser)

“I might look at what we were going to be shooting and suggest that a particular effect should be achieved in-camera or possibly through a makeup effect. There are a lot of potential solutions, so you shouldn’t limit your creative thinking just because the default mindset is to do it in post. And at the very least, what we shoot on set can serve as reference for VFX, so the shot will have a certain grounding to it. I’ve been trained to go out and find real-world references rather than reference another movie for what happens in real life.”

—Marshall Richard Krasser, Visual Effects Supervisor

When possible, rigs were used on set to capture the actors, even when some aspects would be altered in post. (Photo courtesy Marshall Richard Krasser)

Nathan (Robbie Amell) with series creator Greg Daniels on the set representing his Lakeview digs. Spectacular enhanced-reality vistas will be composited into window spaces.

When Nathan considers ending it all by entering Lakeview’s datastream torrent, his “angel” Nora (Andy Allo) drops into his reality to dissuade him in virtual person. Greg Daniels, right, provides direction.

“Nowadays, with the new to-be-determined paradigm owing to COVID, how will we shoot things on set? How will we approach crowd scenes? Do we only shoot with X number of extras and fill in the rest with 3D? I think you’re going to see some shifts in the way things get done. And VFX will be there more often to help tell the story completely and safely. And I think the effects world is more than happy to step up and fill that vacuum.”

—Marshall Richard Krasser, Visual Effects Supervisor

Here, a mockup shot out of doors allowed the capture of natural reflections, and a mockup shot with greenscreen.

“[O]ne problem with the idea of all these self-driving cars is that they behave perfectly – which works against the idea that you’re seeing a real environment, where the vehicles would be changing speeds from the heavy foot on the pedal or somebody braking early. We wound up treating it like cars in a ride at Disneyland… We did try to offset that artificial/video game feel ever-so-slightly by introducing a bit of noise to give the vehicles some road bounce. That together with some camera vibration as well helped take the curse off it.”

—Marshall Richard Krasser, Visual Effects Supervisor

Some of the self-driving cars, auto gadgetry, freeways and vistas of a futuristic L.A. in Upload.

The afterlife scenes featured a degree of stylization with respect to skies and colors but, again, this required a touch that would not distract from the story. “With the episodes that followed, there was a lot of new stuff to come up with, in terms of phenomena unique to Lakeview and other afterlife options, so we brought on our own concept artist while working with production’s in-house art department,” Krasser reveals. “With Upload, you don’t immediately think ‘visual effects extravaganza,’ but there’s a lot of work, from the holographic telephone displays to the little glitches and the rezzing-up as characters come and go from their venue.”

One story point required VFX to generate an extremely low-rez version of their main character. “When we see him transformed into low-poly mode, that was very challenging,” admits Krasser. “If you go back to the early days of computer graphics, there was the music video of Dire Straits’ ‘Money for Nothing’ – which, by the way, was one of my early inspirations for getting into the business. Once Greg selected from among our presented options, it became a matter of how we were going to animate a mouth on these blocky features. The actors were physically shot, so we used that as our guide, painting them out while retaining appropriate characteristics of each performer.

“Then there was what we called the blue room, where Nathan went Heaven-shopping, choosing what options he had outside of Lakeview. That was 12 pages of dialog with 120 shots. To help us stay within budget while accommodating such a volume of work, Greg actually built and photographed a tiny greenscreen set with dolls, comping it with various backgrounds. Production built two versions of the room right next to each other on stage. On one, every object inside was painted blue, while the other looked normal. We put green tape on the edges of the furniture in the blue room so we’d be able to treat those extractions differently from the ones used for the wall backgrounds, letting the viewer see a difference in the geometries when we put various backgrounds over the scene.”

The VFX shot load for the first season of ten episodes came to 1,400, with most of the work done by in Vancouver – though some, including the mouth replacements for a talking dog/therapist, were handled in the Los Angeles office. “You do get a greater sense of consistency when having VFX all under one roof,” Krasser maintains. “Though production did have a small in-house group of artists doing work that didn’t require a great level of expertise or our firepower. We were all constantly trying to find the most efficient solution while delivering the proper spectacle and humor. It was about a year from our initial talks till when we made final deliveries.”

With work now all being done remotely, Krasser finds himself seriously missing the face-to-face aspect of the review process. “Nowadays, with the new to-be-determined paradigm owing to COVID, how will we shoot things on set?” he wonders. “How will we approach crowd scenes? Do we only shoot with X number of extras and fill in the rest with 3D? I think you’re going to see some shifts in the way things get done. And VFX will be there more often to help tell the story completely and safely. And I think the effects world is more than happy to step up and fill that vacuum. We want to help the filmmakers tell the story and do so safely. I think that’s what the future looks like for all of us.”