By IAN FAILES
The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.
Winner of three prestigious Folio Awards for excellence in publishing.
By IAN FAILES
If you’re looking to bring to life a CG human, character or creature with the aid of any kind of performance capture, there’s now a bevy of options at your disposal. Among those are many different motion-capture suits – ranging from optical to inertial systems, as well as ‘faux mocap’ tracking suits – and facial-capture set ups.
VFX Voice asked several mocap vendors and visual effects studios about their various suit and head-cam offerings.
In optical motion capture, infrared cameras pick up the light reflected back from retro reflective markers on the suits (in a ‘passive’ system). ‘Active’ systems allow cameras to sync up to strobing markers on the suits. Each camera sees the marker from a 2D perspective and when all of the 2D files are reconstructed together, a 3D marker in space can be calculated.
Vicon motion-capture suits include a hat, gloves, overshoes and markers that can be glued or taped onto the suits – there are facial markers, too. “The suit has been designed to be comfy to wear, capable of dealing with stunt work and covered in Velcro so we can attach markers to it,” outlines Vicon VFX product manager Tim Doubleday. “The reflective markers are essentially a molded base covered in reflective scotch tape. They can vary in sizes and are small enough to go on fingers.”
Meanwhile, Optitrack also offers a full range of motion-capture and tracking solutions, including a new suit. “The new suits were designed to do what the older suits do – only better,” says OptiTrack chief strategy officer Brian Nilles. “They are now antimicrobial and more breathable than before, offering a better fit and more flexibility, which provides performers with exceptional freedom of movement and comfort over long recording sessions. Our popular X-base markers adhere much better to the new suits, making them nearly impossible to knock off during performance capture, and it allows the performers and the mocap technicians to focus on the performance rather than the tech.”
Also providing optical systems is Fox VFX Lab, which was formerly Technoprops. They’ve made significant innovations in virtual cameras and simul-cams, something Technoprops founder Glenn Derry (now team leader, Fox VFX Lab and VP, Visual Effects, at Fox Feature Films) helped pioneer on Avatar. The Fox VFX Lab suits, gloves and marker patches are made by 3 x 3, with the markers themselves from MoCap Solutions. The shoes are Nike with mocap fabric sewed on. “The mocap suits with the gray fabric and colors make it easy to pick out which actor you are looking at when using reference video to make performance selects,” says Derry. “If everyone is wearing solid black the editor’s job is more difficult. Plus, who doesn’t like festive colors?”
The head rigs from Fox VFX Lab are designed and built in-house. “The original version of our head rigs was designed and manufactured during the production of Avatar, though at the time the camera was singular and standard definition,” describes Derry. “The head rigs have been continuously improved upon feature-wise for the last 13 years. We use them every day on productions ourselves under the most demanding conditions in the business.”
Inertial, or magnetic, motion-capture systems use magnets, accelerometers and gyroscopes all within a contained cable system that tends to zip into some kind of lycra suit. No calibrated volume is required.
Xsens offers suit, sensor/tracker and software solutions. MVN Link and MVN Awinda are Xsens’ main offerings, with the suits using 17 sensors embedded or wirelessly strapped to the body of the performer. “The MVN Link is a full-body, camera-less mocap suit that is connected to a wireless data link and used for high-dynamic movements, like fighting scenes and fast maneuvers,” explains Xsens product manager Hein Beute. “MVN Awinda is the fully wireless version with wireless sensors built into it. It has greater data collection capabilities and can also be used to capture multiple subjects at once.
“The defining characteristic that sets our solution apart from others is the magnetic immunity,” says Beute. “It is the only system that provides magnetic immunity to this level, because of the sophisticated sensor fusion algorithms which took us many years to develop. We can also do multi-level motion capture, which is hard for most other inertial motion-capture systems.”
Perception Neuron also offers a full-body wireless motion-capture system using inertial measurement unit technology. Their Neuron suit comes with a network of straps that house the inertial sensors, known as ‘neurons.’ Full hand and finger tracking is part of the set-up for the 2.0 option. “The system includes the Axis Neuron software, which allows for up to five actors at a time to be recorded or streamed live, and all of the hardware including sensors and straps,” says Perception Neuron’s chief motion-capture technologist, Daniel Cuadra.
“Perception Neuron offers accessibility to all levels of users from professionals to beginners, and is also used by many educational institutions due to its versatility and simple set up,” states Cuadra. “Perception Neuron is also the only motion-capture solution to include full-finger tracking at no extra cost.”
One of the newer entrants in the inertial space is Rokoko, which makes a wireless motion-capture body suit that is right now pitched at indie developers and smaller studios who might not have had access to mocap previously. The Smartsuit Pro contains 19 sensors that are part of a sports textile and mesh suit, with straps positioned wherever there are sensors. Via the gyro, accelerometer and compass tech in the sensors, the system balances the capture data output on a body model on a suit-based hub, which is transmitted through Wi-Fi to a computer or smart device.
“We wanted it to be this personal mocap system,” notes Rokoko founder and CEO Jakob Balslev, “one that you could almost write your own name on, and have there to use anytime. Above all, we wanted something that was intuitive to everybody, something that one person would be able to set up, one person could operate and get started.”
Two of the main players focusing on facial-capture hardware (and which also have software solutions) are Faceware and Dynamixyz. Head-mounted camera systems from Faceware are designed to be used on set, in voice-over or ADR booths and on motion-capture stages, without the need for markers. Faceware also provides a supported software pipeline for getting the data into and out of content creation tools.
Faceware’s main offering is the ProHD Headcam, a fiberglass helmet available in three sizes and fitted with different anodized aluminum boom arms so the user can cover any number of capture situations. The camera is a micro Full HD camera – which also includes an onboard mini light box – and is powered by a separate capture belt for wireless transmission. “The real-time full resolution and full frame-rate transmission allow for the highest quality real-time tracking for live character facial animation, from real-time events to previs,” says Faceware Vice President of Business Development Peter Busch. “The signal can also be recorded for further enhanced tracking and animation.”
Dynamixyz’s facial-capture pipeline involves markerless capture, with video recording and head-mounted camera (HMC) software part of the system. The company provides a custom HMC designed to capture faces (“mostly human,” says Dynamixyz CEO Gaspard Breton, “although we would really like to test it on a pet face one day for fun”). The HMC can be either single view, with one camera facing the actor, or multi-view, with two cameras, one for each side that leaves the field of view unobstructed.
“Our main software Performer enables you to analyze data and re-target them on any 3D model and any rig,” states Breton. “Once the system is trained, it can produce massive amounts of frames throughout batch processing, with minimal rework for amazing results. Our system is also compatible with any body motion-capture system.”
Weta Digital, which has a long history of taking motion-captured actors through to final CG characters – from Gollum in the Lord of the Rings trilogy, to the primates of the Planet of the Apes films, to the characters of Alita: Battle Angel – has now crafted its own custom motion-capture suit.
“On Alita, in particular,” outlines Weta Digital Motion Capture Supervisor Dejan Momcilovic, “we built a custom suit with embedded marker strands that did not allow any marker placement changes, but proved to be very practical, as it was much faster for the performer to get suited up. Head, hands and feet were detachable, so actress Rosa Salazar, as Alita, could have breaks with minimal fuss, which was great given the summer conditions in Texas. The head-cam used was the first stereo cam rig for us, and has since been adopted as our standard rig.”
Momcilovic adds that the suits also fit in with an established workflow at the studio for capturing data. Having their own solution gives Weta Digital more consistency and speed of setup, he says. “Basically, we prepare the suit, hand it off to costumes, and the performers then arrive dressed and ready to go. We only need to attach the body pack and the battery. With our custom-built helmets, we are basically removing the need for additional padding, minimizing the weight and providing better performer comfort.”
ILM, too, has its own custom suit, one that is aimed at providing tracking data of performer movement with low impact. Dubbed IMocap, the suit was first used for the pirate characters in Pirates of the Caribbean: Dead Man’s Chest. “The goal was to get close to optical motion-capture quality, but with a small footprint on set, and able to handle difficult location shooting conditions,” remarks ILM Chief Creative Officer John Knoll.
The suits once had banded tracking dots on them. Using two high definition video cameras, the position of particular joints could be triangulated with software and provide a fast matchmove of the actor. Since then, the suit has evolved and found use in many productions, including Solo: A Star Wars Story, A Quiet Place, Black Panther, Avengers: Infinity War and Aquaman.
IMocap’s markers and dots on the suit have evolved over time, as has the use of custom ILM tools for match-moving and crafting digital characters. “Our most recent version of the IMocap suit, developed for Rogue One: A Star Wars Story, replaces what were colored dots with unique symbols,” describes Senior ILM R&D Engineer Kevin Wooley. “The aim of the symbols is to once again provide more visual distinction and to make it easier to track patterns through blur by allowing the tracker to see different colors in different channels.”