VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


April 20
2021

ISSUE

Web Exclusive

UNVEILING THE TRUTH ABOUT THE LGBTQ PURGE IN WELCOME TO CHECHNYA

By TREVOR HOGG

Images courtesy of Netflix.

Protecting the identities of sources is an integral part of investigative journalism, with the usual techniques being blurring faces, shooting interviewees as silhouettes and modulating voices. HBO Documentary Films’ Welcome to Chechnya, which exposes the systematic persecution of the LGBTQ community in Chechnya, introduces an innovative approach combining machine learning and face replacements that earned a nomination for Outstanding Supporting Visual Effects in a Photoreal Feature at the VES Awards, and became the first documentary to make the Oscar shortlist for Best Visual Effects.

“There were so many challenges,” states filmmaker David France, who received an Academy Award nomination for How to Survive a Plague. “The first was to talk to the people who needed to be anonymous about my desire to film them. We came up with a strategy working together about how to protect the digital files and how to move them safely out of the country and how to erase the cards from the cameras. We were overriding those cards so in case we lost control of one of those,  data couldn’t be harvested. Then I talked with the folks about wanting to find some way to disguise [people to be filmed anonymously] that doesn’t distract from their humanity. I didn’t know what I was talking about, but I was honest with them about not knowing what I was talking about. We agreed that we would come back to them. They gave me a conditional release. I would come back to them for these security reviews, and they would ultimately give the sign-off. I left with the footage well-protected and began that work once we got back to New York.”

The process of discovery began when producer Alice Henty (Buck) called veteran visual effects supervisor Johnny Han (The Nevers), who became the Facial Capture Supervisor on the project. “I said to Alice, ‘There are things being done now in the computer graphic community where we rely on whole other processes such as machine learning to put the likeness of one person’s face onto another,’” states Han. “I phoned Ryan Laney [Spider-Man 3] and asked him if he wanted to help me.” Capturing every possible facial angle and distance was not possible. “When watching the film and breaking down the shots, we realized that there was a certain comfortable distance for the average person when filming someone with their iPhones,” adds Han. “We decided upon two different lengths: a macro and a wide.”

Director David France

“There were so many challenges. The first was to talk to the people who needed to be anonymous about my desire to film them. We came up with a strategy working together about how to protect the digital files and how to move them safely out of the country and how to erase the cards from the cameras. We were overriding those cards so in case we lost control of one of those, data couldn’t be harvested.”

—David France, Director

“I worried having no experience in this that we might be flirting with the uncanny valley,” admits France. “I reached out to Thalia Wheatley, who is a professor at Dartmouth College. She has a neuro-psych lab there, and I actually enrolled in a study to test the various approaches that Ryan had been developing to see if they work. In fact, the approach that we ultimately settled on scored a little higher than an uncovered face in empathy and in one’s ability to connect with the characters. We had been developing this idea called ‘the tell’ where there is a soft halo around the face that showed the viewer that something had been done to it.” The facial capture sessions with the LGBTQ activists took place in New York City with a nine-camera array. “We collaborated on the casting with David’s team to make sure within his pool of activists that we could find jawlines and general body composition that matched,” states Ryan Laney, who served as the Visual Effects Supervisor. “There were a lot of challenges in that. It was important for us to get something that felt natural when the veil was in place.”

There were 122 distinct lighting setups split across 23 subjects. “We went from the biggest lighting setup, which needed the most lights, and dwindling it as the five days went on, and returning the lights to the lighting company as we were done with them,” remarks Han. “I would sit with the activists and on a secret laptop,  take them through the scene that they were about to do. In the most delicate way, I talked them through what emotion that we were trying to get. You can’t just tell someone, ‘I want you to raise your right eyebrow and look sad.’ You have to feel it. To make it even harder for the activists, we had an on-set language coach feed them a pangram, which is designed to take the mouth through all of the different phonetic shapes. However, we couldn’t use American English mouth shapes. It had to be specific to Chechnya.”

With cameras not being allowed in airports in Russia and not wanting to draw attention to the production, regular film cameras were not a viable option. “We had rolling shutter issues from these consumer-grade cameras and low light where the grain was more than the signal,” remarks Laney. “We used some computer vision to find faces, so we got reasonable camera tracks from that. But there was a lot of manual tracks and de-graining sessions to mitigate these low-quality issues that came with some of the footage. In some of these cases where the grain was really prevalent, it sometimes showed up as micro-expressions, so the face would do strange things. It was interesting in how we dealt with each one of these complications so that we got a truthful representation of what was there originally.   We ended up being all 2D. It was Nuke, After Effects, and edited in Avid. We relied more on talent than tools in this particular case.”

“After Ryan did his magic, we brought the scenes back to the subjects so they could see the entire run, and that is when we discovered that jacket was a gift, so we would have to change the jacket, or the writing on that shirt was going to give something away, or that piece of jewelry was a family heirloom,” remarks France. “We even found that one person’s head shape was a telltale sign of his identity. We worked on every part of the frame, like a room that would suggest what state, country or city that they’re in. Anytime we have an establishing shot looking out the window, that was often a different country all together. We wanted it to be confounding for anyone trying to use the film to find people.”

Visual Effects Supervisor Ryan Laney

Shooting in Russian airports is illegal, so the footage was captured using consumer-grade cameras, which resulted in having to correct issues with grain, tracking and rolling shutters.

“We even found that one person’s head shape was a telltale sign of his identity. We worked on every part of the frame, like a room that would suggest what state, country or city that they’re in. Anytime we have an establishing shot looking out the window, that was often a different country all together. We wanted it to be confounding for anyone trying to use the film to find people.”

—David France, Director

Clothing and locations were digitally augmented to make identification of those interviewed even more difficult

“It was all done in service of truth. We did not create fiction. We created the ability for this truth to be told publicly through the use of visual effects. It’s about giving power to people who have been powerless. It’s about giving back humanity to people who have been chased into the shadows. It’s a phenomenal use of the technology that we have enjoyed for so many years in Hollywood filmmaking being put it into the defense of humanity for the first time.”

—David France, Director

An unveiling takes place as to the true identity of Grisha. “Maxim Lapunov had already done that press conference before we began this work, so there was a discussion initially about not covering him,” explains France. “But people needed to know how much risk he was already facing, so we decided to cover him until that press conference. Then we debated at what point in the press conference do we reveal him. [After a test screening] we decided to do it when a woman suddenly gives him a microphone and calls him Maxim Lapunov. Ryan tried a number of other ways to do it. There was just so much magic in Grisha’s brown eyes becoming Maxim’s beautiful blue eyes that invited you to feel the peeling back of the skin.”

The term deepfake does not apply to Welcome to Chechnya. “It is a similar style transfer of a deep-learning idea,” notes Laney. “But we think that deepfakes are inherently non-consensual, and we were careful about being open to everyone as to what was going on.” France is in agreement. “It was all done in service of truth.

We did not create fiction. We created the ability for this truth to be told publicly through the use of visual effects. It’s about giving power to people who have been powerless. It’s about giving back humanity to people who have been chased into the shadows. It’s a phenomenal use of the technology that we have enjoyed for so many years in Hollywood filmmaking being put it into the defense of humanity for the first time.” 

“We collaborated on the casting with David’s team to make sure within his pool of activists that we could find jawlines and general body composition that matched. There were a lot of challenges in that. It was important for us to get something that felt natural when the veil was in place.”

—Ryan Laney, Visual Effects Supervisor

At a press conference, Grisha is revealed to be Maxim Lapunov.

Members of the LGBTQ community in New York City lent their faces to be used as veils.

A nine-camera rig was utilized during a five-day facial-capture shoot held in New York City.

There were 122 distinct lighting setups split across 23 subjects.

A render using the learned model.

An on-set language coach provided the activists with a pangram that was designed to take the mouth through all of the different phonetic shapes that were specific to Chechnyan.

Machine learning is dependent on quality data collection.

Maxim Lapunov is paired with his face-double.

The facial veils were applied by using 2D techniques in Nuke and Adobe After Effects.

A facial halo effect was created to emphasize that the identity of the interviewee had been altered.

One of the initial ideas was to use the roto animation technique found in A Scanner Darkly (2006).

Share this post with

Most Popular Stories

The Miniature Models of <strong>BLADE RUNNER</strong>
02 October 2017
Exclusives, Television/ Streaming
The Miniature Models of BLADE RUNNER
In 1982, Ridley Scott’s Blade Runner set a distinctive tone for the look and feel of many sci-fi future film noirs to come, taking advantage of stylized production design, art direction and visual effects work.
How to Start a <strong>VFX Studio</strong>
01 October 2019
Exclusives, Television/ Streaming
How to Start a VFX Studio
Four new VFX studios (CVD VFX, Mavericks VFX, Outpost VFX, Future Associate) share their startup stories
Converting a Classic: How Stereo D Gave <strong>TERMINATOR 2: JUDGMENT DAY</strong> a 3D Makeover
24 August 2017
Exclusives, Television/ Streaming
Converting a Classic: How Stereo D Gave TERMINATOR 2: JUDGMENT DAY a 3D Makeover
James Cameron loves stereo. He took full advantage of shooting in native 3D on Avatar, and has made his thoughts clear in recent times about the importance of shooting natively in stereo when possible...
The New <strong>Artificial Intelligence</strong> Frontier of VFX
20 March 2019
Exclusives, Television/ Streaming
The New Artificial Intelligence Frontier of VFX
The new wave of smart VFX software solutions utilizing A.I.
THE PEARL: THE SUPER ALIEN MODELS OF<strong> VALERIAN</strong>
02 August 2017
Exclusives, Television/ Streaming
THE PEARL: THE SUPER ALIEN MODELS OF VALERIAN
Among the many creatures and aliens showcased in Luc Besson’s Valerian and the City of a Thousand Planets are members of the Pearl, a beautiful...