VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


September 04
2018

ISSUE

Web Exclusive

The Puppet Masters Behind THE HAPPYTIME MURDERS

By IAN FAILES

All images copyright © 2018 STXfilms.

A world in which humans and puppets co-exist is the world created in Brian Henson’s The Happytime Murders, an adults-only detective romp that takes puppeteering – and visual effects – to a new level.

Stargate Studios’ Sam Nicholson oversaw VFX on the film as Visual Effects Supervisor, with work ranging from classic rod and puppeteer removal, to intricate compositing of greenscreened puppets, to the inclusion of fully CG puppets on screen.

VFX Voice asked Nicholson how he managed the work for the Jim Henson Company production.

Watch the RESTRICTED trailer for The Happytime Murders.

“It’s some of the most complex work that we’ve ever done because, in one shot you could have a combination of principal photography done at 8K and then puppets on greenscreen being puppeteered by four puppeteers for each puppet. So if you have six puppets in a scene, you’ve got 24 puppeteers operating them, and these are all people dressed in green suits.”

—Sam Nicholson, Visual Effects Supervisor, Stargate Studios

VFX Voice: This seems like a very different kind of visual effects assignment. What was involved in it from your end?

Sam Nicholson: This was a big visual effects film. We’re delivering over 750 shots for the show. It’s some of the most complex work that we’ve ever done because, in one shot you could have a combination of principal photography done at 8K and then puppets on greenscreen being puppeteered by four puppeteers for each puppet. So if you have, say, six puppets in a scene, you’ve got 24 puppeteers operating them, and these are all people dressed in green suits. And then you have all the rod removal of those puppets, and then you have avatars or CG puppets that are standing right next to the real puppets.

VFX Voice: What would determine whether a puppet would be achieved as a greenscreen element or as an avatar, or in some other way?

Nicholson: The general rule of thumb was that everything that was possible to shoot waist-up we did practical, but even then we tried to make it easy on the puppeteers. So rather than have a puppeteer lying down on the ground and operating a puppet walking next to a human being, we’d just let them be in frame and get a much better performance that way. That meant we had a lot of puppeteers in green outfits with rods and everything just all over every shot. Of course they had to be removed, and everything had to be reconstructed behind them.

Greenscreen puppeteers work on a greenscreen stage to provide elements for a street scene.

“We had a lot of puppeteers in green outfits with rods and everything just all over every shot. Of course they had to be removed, and everything had to be reconstructed behind them.”

—Sam Nicholson, Visual Effects Supervisor, Stargate Studios

There are a couple of really big shots with 20 or 30 puppets all in the scene at Santa Monica beach in broad daylight with avatars and actors. We shot it all wild on a Technocrane, analyzed that move, then scaled that for motion control, and shot the greenscreen puppets that needed to go into the scene on a separate set using scaled motion control that would match the full-scale scenes. There was a lot of complicated programming that went into that.

VFX Voice: How did you approach scenes that required CG puppets, or avatars, as you call them?

Nicholson: Anytime you see a full-height puppet, it’s generally an avatar. That avatar has to be an exact match to the look and feel and animation of the real puppet itself. You have hair and fabric, and you need to match the way the light absorbs into felt. Skin is actually pretty easy compared to puppet felt!

The close-up avatars were scanned and had a complete texture model done, and others were done with photogrammetry. Some of the puppets on set that had their heads explode or get shot, we’d just do a bunch of photogrammetry of it so that we have a digital duplicate to match to the live action.

Phil Phillips is puppeteered on the greenscreen set.

“The close-up avatars were scanned and had a complete texture model done, and others were done with photogrammetry. Some of the puppets on set that had their heads explode or get shot, we’d just do a bunch of photogrammetry of it so that we have a digital duplicate to match to the live action.”

—Sam Nicholson, Visual Effects Supervisor, Stargate Studios

We also did some motion capture of the puppets on set with Xsens MVN sensors on the puppets. And we had a puppeteer who was operating the main character, Phil Phillips, wearing a suit so that we could capture his leg movements. I wanted to capture all of Phil’s movements because, keep in mind that the shadow reconstruction is very important here.

Generally, the floors are all built up four feet high on a puppet set because puppets are operated above people’s heads. So even when you do something as simple as walk through a door into a room, the door is split in half so that the puppeteer can articulate his body through the door, but you have only the upper part of the door. You have the upper part of the torso, the puppet grabbing the doorknob, but then everything below the door knob is puppeteers and junk. So sometimes you have three people trying to fit through a doorway at the same time. It’s split, and there is no floor at all, so you have to reconstruct the floor, reconstruct the door, and then take the motion capture that we got on the day and add it and blend it into the puppet.

Puppeteer Bill Barretta (left) and director Brian Henson on the set of The Happytime Murders.

“Generally, the floors are all built up four feet high on a puppet set because puppets are operated above people’s heads. So even when you do something as simple as walk through a door into a room, the door is split in half so that the puppeteer can articulate his body through the door, but you have only the upper part of the door.”

—Sam Nicholson, Visual Effects Supervisor, Stargate Studios

VFX Voice: Puppets move in a certain way because of the way they’re built – how did you match that with any CG movement?

Nicholson: Yes, one of the interesting things is that, when you think about it, if a puppet is on your wrist or on your arm and you rotate your wrist one way, your wrist only rotates one way for a certain amount. So when a puppet turns its head, depending on which hand the puppeteer’s using, the puppet can look around 180 degrees in one direction, but it might only be 30 degrees in the other direction. So we really had to match that kind of movement.

VFX Voice: What was your toughest shot?

Nicholson: There’s some shots in there with multiple puppets in full daylight and the camera’s moving, and you’ve also got to remember the whole point of the scene is that it needs to be funny. One of the funniest scenes is in the porn shop – the octopus milking a cow has got to be one of the classics of all time. But I mean, you’ve got to make that look spontaneous. Where do you find that in the rule book? I just don’t know.

Another greenscreen element featuring Phil Phillips.

Share this post with

Most Popular Stories

CHECKING INTO HAZBIN HOTEL TO CHECK OUT THE ANIMATION
16 July 2024
Exclusives, Film
CHECKING INTO HAZBIN HOTEL TO CHECK OUT THE ANIMATION
Animator Vivienne Medrano created her series Hazbin Hotel which has received 109 million views on her VivziePop YouTube Channel.
LIGHTWHIPS, DAGGERS AND SPACESHIPS: REFRESHING THE STAR WARS UNIVERSE FOR THE ACOLYTE
30 July 2024
Exclusives, Film
LIGHTWHIPS, DAGGERS AND SPACESHIPS: REFRESHING THE STAR WARS UNIVERSE FOR THE ACOLYTE
Creator, executive producer, showrunner, director and writer Leslye Headland is the force behind The Acolyte, which occurs a century before the Star Wars prequel trilogy.
SUMMONING CREATIVE VFX TO HEIGHTEN REALITY IN THE SYMPATHIZER
09 July 2024
Exclusives, Film
SUMMONING CREATIVE VFX TO HEIGHTEN REALITY IN THE SYMPATHIZER
Park Chan-wook was an ideal choice as a co-showrunner, director and writer to create a seven-episode adaptation of The Sympathizer for HBO.
TONAL SHIFT BRINGS A MORE CINEMATIC LOOK TO HALO SEASON 2
23 July 2024
Exclusives, Film
TONAL SHIFT BRINGS A MORE CINEMATIC LOOK TO HALO SEASON 2
There is an influx of video game adaptations, with Paramount+ entering into the fray with the second season of Halo.
FILMMAKER PABLO BERGER MAY NEVER STOP HAVING ROBOT DREAMS
06 August 2024
Exclusives, Film
FILMMAKER PABLO BERGER MAY NEVER STOP HAVING ROBOT DREAMS
The Oscar Nominated Spanish-French co-production Robot Dreams deals with themes of loneliness, companionship and people growing apart – without a word of dialogue.