23 January 2024
Animation
By BARBARA ROBERTSON
By BARBARA ROBERTSON
Studios creating a live-action blockbuster might cite more than 1,000 visual effects shots in the film, perhaps even 2,000, including some sequences in which everything – environments, effects, actors – is digital. These sequences are like animated films within the live-action film. But people rarely think in terms of the opposite – that is, they rarely think of animated features in terms of “visual effects.”
“When people who don’t know the animation industry hear I’m a visual effects supervisor, they just ask about effects,” says Dave Walvoord, Visual Effects Supervisor for DreamWorks Animation’s How to Train Your Dragon: The Hidden World, the third and final film in the popular and critically acclaimed franchise. “I have to explain that ‘visual effects’ is not just blowing things up – that in Planet of the Apes, Caesar is a visual effect – and it’s like that here, too.”
In fact, a visual effects supervisor on an animated feature typically oversees everything except editorial, layout, previs, story and the animated performances, as did Walvoord for The Hidden World. The first film of the franchise, How to Train Your Dragon, won an Annie in 2011 for Best Animated Feature, three VES Awards (for Animated Character, Animation and Effects Animation), and received an Oscar nomination for Best Animated Feature. How to Train Your Dragon 2 won an Annie for Best Animated Feature, received an Oscar nomination for Best Animated Feature, and four VES nominations.
Actor Jay Baruchel voices the main character, Hiccup, a young Viking who in the first film rescued and befriended an injured black dragon, a Night Fury he dubs Toothless. Hiccup bonds with Toothless, and ultimately the pair leads their communities – human and dragon – to live in harmony. In this third installment, Toothless meets a white dragon, the Light Fury.
“The challenge for Hiccup is that he wants Toothless to find a mate,” says Simon Otto, head of character animation for all three Dragon films. “But the only way Toothless can fly is with Hiccup on his back [because of an injury from the first film]. And the only way to be with the Light Fury is when Hiccup is not there.”
Dean DeBlois directed all three films. Walvoord was Visual Effects Supervisor for Dragon 2 and Dragon 3, managing teams doing modeling, rigging, character effects, digimattes, effects, simulation, lighting, rendering, compositing, stereo, and so forth. DreamWorks Animation teams use a combination of commercial software and proprietary software tools that have evolved since the first Dragon film took flight.
“We changed a lot of technology for this show,” Walvoord says. Those changes include a new character effects pipeline and the adoption of open source USD (Universal Scene Description), which opened up their ability to handle complex scenes. The most dramatic change, though, was the studio’s new proprietary ray tracer “Moonray” along with a new “Moonshine” shader platform, both used for the first time on this film.
“Considering how much technology we changed, that we were using a ray tracer for the first time, and that we built an entirely new pipeline in six months to a year, the surprising thing is that this wasn’t the hardest movie I’ve made. It all came together well and quickly.”
—David Walvoord, Visual Effects Supervisor
“Considering how much technology we changed, that we were using a ray tracer for the first time, and that we built an entirely new pipeline in six months to a year, the surprising thing is that this wasn’t the hardest movie I’ve made,” Walvoord says. “It all came together well and quickly. We had giant machines with 48 cores, 196 gigabytes of RAM, and a ray tracer that could take advantage of all that. We had more sophisticated algorithms and software. But we didn’t have a very good schedule. So we started to work more like a visual effects movie.”
Although artists creating 3D-animated films use many of the same tools and techniques as those creating visual effects for live-action films, their process often resembles the linear pipeline developed for 2D-animated films rather than methods used in visual effects studios.
“In visual effects, it feels like every shot is its own problem to solve – not entirely true, but more so than in animation,” Walvoord says. “The 2D legacy sends you down a linear path with an economy of scale that visual effects don’t have: If there are two characters in a set, every shot can be approached in the same way. But we didn’t do that. We approached every shot looking at how we could make it the best it could be. We animated while we were building sets. We were lighting while were surfacing. All the departments collaborated heavily. Our new pipeline and a common toolset made it easy to pass data back and forth and work in parallel.”
Helping the artists embrace a live-action methodology was Oscar-winning cinematographer Roger Deakins, who was a visual consultant on Dragon 2 and again on this film.
“On the second Dragon, he told us to stop worrying about continuity, and we took it to heart,” Walvoord says. “Like a cinematographer on set, we just lit the shot. We realized that it’s much more interesting to work this way.”
Dragon 3 is the 12th film for which Pablo Valle has been a lighting supervisor. On this film, he supervised 40 lighting artists, organizing them into four teams. Each team handled approximately 10 of the film’s 40 sequences.
“This film is easily the pinnacle of what I’ve done, both in meeting our artistic goals and in our technical achievements,” he says. “Every day we wondered, ‘How are we going to do this?’ It was an absolute blast.”
The lighting team leaned on Deakins to help them elevate the look of this film without deviating too far from the design language of previous films.
“[Cinematographer] Roger [Deakins] has a unique way of telling stories through images that may not be in line with what people expect from animated films. Animated films are often very bright. They leave little for the imagination. But Roger is not like that. He likes bold images through pushed composition, through using minimal lighting sources, and with a palette more like visual effects and live-action films than animated films. We embraced that everywhere we could.”
—Pablo Valle, Lighting Supervisor
“Roger [Deakins] has a unique way of telling stories through images that may not be in line with what people expect from animated films,” Valle says. “Animated films are often very bright. They leave little for the imagination. But Roger is not like that. He likes bold images through pushed composition, through using minimal lighting sources, and with a palette more like visual effects and live-action films than animated films. We embraced that everywhere we could.”
The most spectacular lighting happens in the Hidden World, a magical dragon lair with miles of caverns made with bioluminescent mushrooms, crystals, water, waterfalls and mist. It pushed the new ray tracer to its limit.
“The Hidden World is made of all the things that are hard to do, but thankfully our lighting leads came up with clever ways to package lighting rigs into plug-and-play things that could be dropped into sequences,” Valle says “We had a library of light types and responses and could deal with volumetrics in a collaborative way. Before, we sometimes needed eight weeks to see an image. On this film, we could sometimes turn around complex images in a day.”
The packaged lighting assets meant the lighting artists could place sources where they needed lights – as would a DP on a live-action set – rather than, as more typically on an animated feature, having the light sources predetermined in layout or by effects simulations.
“For example, when the characters are camping in the middle of a forest, rather than having layout artists place sources, we could package lamps, torches and volumetric fire into an asset that the lighters would move around the set to the best places,” Valle says. “We placed the lanterns for some shots in thick fog. Also, we would light to lines of dialog in animated shots. We learned that when lighting is dependent on sources, the sources should be placed in lighting, not before, and/or we should be involved from the beginning.”
“Hiccup couldn’t be photoreal, but he had to look better than in Dragon 2. We came up with a way to add detail and create a believable character that reacts to light better, and still stay away from the uncanny valley.”
—David Walvoord, Visual Effects Supervisor
“Our algorithms are better, as is the light transport simulation and our surfacing. We even introduced peach fuzz which we’d been cautious about. It softened the characters and helped us pull away from CG.”
—David Walvoord, Visual Effects Supervisor
Dragon 3 is, in many ways, a traveling film – Hiccup’s villagers, threatened by people who want their dragons, move from place to place through the film. But the three main environments are the Isle of Berk where the previous films largely took place, New Berk to which the characters retreat, and the Hidden World. Berk and New Berk have roots in Nordic reality with cliffs, oceans and villages. The Hidden World pushes further.
“In the Hidden World there are obsidian rock towers covered in bioluminescent coral,” Walvoord says. “We built 22 spires and instanced them hundreds of times over a three-mile span. On top, we have 140 million pieces of mushroom coral. It wasn’t easy on the ray tracer, but because we did it with instancing, it was possible.”
A new procedural scattering system called “Sprinkles” gave surfacers tools for set dressing.
“They basically painted geometry,” Walvoord says. “We used Sprinkles for ferns, trees, rocks, bark on trees, flowers – any kind of set dressing – and coupled that with the surfacers doing texturing at the same time.”
In one scene, Hiccup plows through a forest, pushing aside “Sprinkled” grass, ferns and anything else in the way. Birds and bugs move aside.
“If you watch animated movies closely, you often see clear paths for characters to neatly walk through,” Walvoord says. “We wanted to bring the environments into the world of the characters. Rather than having clear paths in a forest, animators started whacking.” Animators mimed the action using proxy plants. Then surfacers dressed the set and character effects artists ran the simulations.
Simon Otto organized his team of 55 animators with seven supervising animators responsible for main characters and a few leads for smaller characters.
“They would own the characters,” Otto says. “They knew where the libraries were, how to work with the rigs. They led the decision-making on the acting and defended their characters through the production.”
Animators on Dragon 2 were the first to use the studio’s Premo Animation System. Using Premo, animators can scrub through and work in real time with multiple shots and access entire sequences. In 2018, Alex Powell, Jason Reisig, Martin Watt and Alex Wells received Academy Certificates in the Scientific and Engineering Awards for developing the system.
“Of course, Premo has evolved since Dragon 2,” Otto says. “The rigs are incredibly sophisticated. We can create more complex, detailed performances. And, as we work in real time with geometry and surfaces, we can see more details. In essence, though, it’s the same basic setup. The big change was on the lighting side.”
“This film is easily the pinnacle of what I’ve done, both in meeting our artistic goals and in our technical achievements. Every day, we wondered, ‘How are we going to do this?’ It was an absolute blast.”
—Pablo Valle, Lighting Supervisor
“In the Hidden World there are obsidian rock towers covered in bioluminescent coral. We built 22 spires and instanced them hundreds of times over a three-mile span. On top, we have 140 million pieces of mushroom coral. It wasn’t easy on the ray tracer, but because we did it with instancing, it was possible.”
—David Walvoord, Visual Effects Supervisor
The tricky question for the lighting artists and shader writers was in making the characters look older and better, yet still look like they are part of the same world as before. “Hiccup couldn’t be photoreal, but he had to look better than in Dragon 2,” Walvoord says. “We came up with a way to add detail and create a believable character that reacts to light better, and still stay away from the uncanny valley.”
The team changed the subsurface model they had used, changed the film LUT, and changed the shaders to have physically-based responses.
“Our algorithms are better, as is the light transport simulation and our surfacing,” Walvoord says. “We even introduced peach fuzz which we’d been cautious about. It softened the characters and helped us pull away from CG.”
The trickiest dragons to light were the black Toothless and the white Light Fury.
“With a black dragon, his specular response, the shiny skin, determines how he looks,” Valle says. “Light Fury has an iridescence that shines when light hits her. Roger [Deakins] helped us carefully construct compositions when both characters were on set and with clever and subtle ways to have Toothless shine more.”
As the characters in the film move toward a new world, so did the crew creating this film. New tools made it possible to work more collaboratively in ways that resembled the way in which visual effects studios work on live-action films.
“With our schedule, we didn’t have much of a choice but to collaborate,” Walvoord says. “But our new pipeline made it easy to pass data back and forth and work in parallel. That can be stressful for the artists. But for me, it was exciting. I think it’s better. It gave us the ability to have happy accidents and the flexibility to invent on the fly. We ran with it. Everything about this film was a lot more fun.”
Head of Effects Lawrence Lee led a team of approximately 30 artists who created magical effects for How to Train Your Dragon: The Hidden World.
“We had traditional effects – fire, water, and destruction – on this movie, but we had three big new things: the entrance to the Hidden World is a set made of water and we had many waterfalls in the New Berk environment, we gave the Light Fury dragon the ability to become invisible, and we gave a bad dragon acidic fire.”
The entrance to the Hidden World is a four-kilometer wide hole in the ocean into which water pours thanks to a group of effects artists led by Baptiste van Opstal. The falling water is a volume-preserving particle simulation and a fluid simulation.
“We had 16 machines simulating simultaneously,” Lee says. “The result was a big level set with three billion voxels. We rendered them directly as iso surfaces all in one pass.”
The entrance leads to an underground cavern filled with waterfalls. For one large shot the artists generated 600 waterfalls from 15 basic waterfalls.
“We had a cool tool that let us set dress the waterfalls everywhere Lee says. “It had physics built into it so the water collides with the terrain, and the artists could change the speed of the water. We gave them to lighting as a gizmo. We can now give lighting artists tools for effects we traditionally do when it’s more efficient. If they want to add haze they can do it themselves. It works out really well – it feels more like a small visual effects shot than feature animation production.”
To make the Light Fury invisible, the artists turned her scales into mirrors. She blows a fireball. As she flies through it, her scales heat up and become reflective. And she emerges “invisible.” As she cools, she becomes visible again.
“We had to choreograph reflections on the mirrors to have recognizable objects show on the surface,” Lee says. “We didn’t always get that out of the box because of her angle, so every shot became a hero shot.”
She becomes visible again using a simulation based on cellular automata patterns developed by DreamWorks Animation CG Specialist Amaury Aubel, who also led work on the acid fire, a liquid simulation that erodes any hard surface it hits.
;
