Effects Add Extra Beats to Music Videos

By IAN FAILES

For VFX artists, music videos can be a very different kind of project compared to film or television. The schedules are often tighter, the budgets lower and the crews smaller. But the amount of work involved in making VFX for music videos, also known as promos, is rarely lower.

Indeed, many promos run like short films and often include a range of diverse, if not over the top, imagery. To get a sense of the state of play in this area of effects, VFX Voice tracked down studios and individual artists behind some of the most captivating VFX imagery in promos in recent years.

FUN AND FANTASTICAL

Music videos tend to showcase a confined story. In the case of DJ Snake’s “Magenta Riddim,” a firefighter visits India to demonstrate how to extinguish a small blaze. But things quickly escalate from this simple story, and visual effects were used to tell a firefighting story that channels the typical over-the-top action scenes of Tollywood, the south India film industry.

Designer Tal Baltuch led the visual effects work for the music video, taking cues from directors Gal Muggia and Vania Heymann. “Vania and Gal wanted to apply the energy of Tollywood to this tale of heroic firemen,” says Baltuch. “It needed to be epically entertaining and obviously not bound to physics as we know it.”

Each VFX shot in the promo would make use of smoke and fire and the addition of practical elements filmed at Ramoji Film City in India. “There was a lot of fire, smoke and flying cars on the set, but not enough,” notes Baltuch. “Each of the firefighting shots ended up being a composite of the shot action and special effects, footage of volcano eruptions, fire, smoke, and some additional 3D and simulations.

“Ideally,” Baltuch adds, “we might have had months to work with a large group of 3D artists simulating each shot. But that was not the case. And I think both Vania, who directed and did the VFX with me, and myself prefer having more direct control over the result and favor using compositing techniques rather than 3D simulations when possible.”

A greenscreen element for a shot in DJ Snake’s “Magenta Riddim” promo. (Image courtesy of Tal Baltuch)

The final shot involved smoke simulations and compositing of the live-action shots. (Image courtesy of Tal Baltuch)

“[Directors] Vania [Heymann] and Gal [Muggia] wanted to apply the energy of Tollywood to this tale of heroic firemen. It needed to be epically entertaining and obviously not bound to physics as we know it.”

—Tal Baltuch, Designer, Visual Effects, DJ Snake’s “Magenta Riddim

A mix of practical effects on location served as the base of many shots for “Magenta Riddim.” (Image courtesy of Tal Baltuch)

The final composite. (Image courtesy of Tal Baltuch)

Maroon 5’s “Wait” breakup storyline includes a scene where one character’s face is made up of paint. (Image courtesy of Timber)

Adam Levine is unwound as a huge length of yarn, simulated by Ingenuity Studios. (Image courtesy of Timber)

Practical pyro effects were augmented with CG missiles and composited explosions. (Image courtesy of Timber)

The Maroon 5 music video for “Wait” features another seemingly innocuous story, this time about a breakup. That is, until strange events start to happen, courtesy of visual effects studio Timber. One character’s face disintegrates into paint, weird things take place underwater, missiles make a mark in a car junkyard and, finally, the lead singer, Adam Levine, gets unwrapped as endless strands of yarn.

For the ‘face of paint’ shots, Timber worked with production designer John Hammer to build a mask out of paint that could be manipulated. “We shot extra passes of the actress Alexandra Daddario with it on her face with Velcro straps,” explains Timber Creative Director Jonah Hill. “We also got performance takes with her without the paint mask. In digital composite we carefully combined them so it looked like one pass.”

Underwater scenes were handled by filming swimmers in a water tank, and then duplicating them, with Levine shot separately in a pool. Meanwhile, practical pyro explosions drove how the junkyard shots came together, with this sequence also showcasing a unique transition from underwater, inside the car, and out to the yard.

Finally, the yarn shots made use of procedural simulations that unraveled Levine and the house around him. Says Hill: “We used Houdini to do the effects, but actual simulations were secondary, really only used for the wiggling yarn pieces that stretched across the room. This was a challenging sequence, but we’re happy with the decisions we made.”

“All the Stars,” from Kendrick Lamar and SZA, takes the fun and fantastical even further, mixing in starfields, fields of waving hands, panthers and more for an especially lavish production. BUF was brought on to produce the visual effects. “We received a very detailed moodboard from the director Dave Meyers,” says BUF Executive Producer Loris Paillier, “and we had some calls to make sure we understood what he had in mind.”

Shots of a boat sailing through a myriad of waving hands relied on greenscreen photography of Lamar and motion-control plates, with only a small group of people for the crowds. “We put the images of the arms on simple 3D models and we put them on a 3D ocean to create the animation,” details BUF CG artist Marion Eloy and VFX Supervisor Dominique Vidal. “Then we worked a lot on the sky part with different pieces of clouds.”

The panther scenes also relied on motion control, with a real panther shot multiple times in separate passes (Lamar was also filmed separately), then combined. Equally majestic were scenes of SZA among a starfield. She was filmed greenscreen, with the starfield particles inhabiting the space around the singer. “The design and distribution of all the stars – in fact, they are made of little bulb lights – was specific to each shot in order to avoid being hit by SZA during the dance,” says Vidal. “The other tricky point was to replace the ground with a reflective surface, thus we needed to rebuild SZA’s reflection, missing in the original plate. At the end, meticulous color grading has been done during the compositing stage to simulate a consistent lighting on SZA in this colorful world.”

Greenscreen boat plate. (Image courtesy of BUF)

A final BUF shot from “All the Stars” by Kendrick Lamar and SZA features Lamar in a boat amongst a sea of hands. (Image courtesy of BUF)

Greenscreen crowd plate. (Image courtesy of BUF)

Greenscreen hands plate. (Image courtesy of BUF)

Simplified 3D hand models onto which real hands are projected. (Image courtesy of BUF)

A panther scene made use of multiple real greenscreen panther plates composited together. (Image courtesy of BUF)

SZA appears in a starfield orchestrated by BUF. (Image courtesy of BUF)

Weirdcore’s visuals for the Aphex Twin single “T69 Collapse” were made up of photogrammetry elements stitched together in unusual ways. (Image courtesy of Weirdcore)

A cocoon forms in Die Antwoord’s “Alien.” JAMM completed the VFX for the promo. (Image courtesy of JAMM)

“We’re used to doing complex creature work, but the cocoon was a new type of challenge. It was important to [band member and director] Ninja that the cocoon looked very real, but also unlike anything people had seen before. We based the material of the cocoon on different molds and fungi to ground it in reality. Then we simulated it and allowed it to evolve and grow on its own into a very detailed and beautiful structure.”

—Zak DiMaria, CG Artist, Die Antwoord’s “Alien”

Maroon 5’s promo for “Three Little Birds” saw them perform against greenscreen. (Image courtesy of Ingenuity Studios)

Ingenuity Studios turned the band members into different forms, often with simulated FX. (Image courtesy of Ingenuity Studios)

TRIPPING OUT

Music can, of course, have its own hallucinogenic effects, and there are plenty of music videos that provide matching stunning imagery. Take, for example, Aphex Twin and the “T69 Collapse” single. The musician has had a long-term collaboration of intense visuals with artist Nicky Smith, also known as Weirdcore. For the “T69 Collapse” music video, the viewer races through obscure cityscapes and other terrains. Smith, who describes his visually impactful work as evoking a “punk attitude,” generated the imagery by acquiring geometry using photogrammetry with Agisoft’s PhotoScan.

Unique flickering and oscillating looks in the video were achieved using After Effects, Cinema 4D and Max/MSP. The music was synchronized to the visuals using a combination of AniMidi for Cinema 4D, Trapcode Sound Keys and Greyscalegorilla’s Signal. Smith says those visuals came to him from “listening to the track or the artist’s tracks on a loop. I also checked my many art books while listening to the music to trigger ideas.”

Perhaps just as trippy is Die Antwoord’s “Alien,” a promo that sees band member Yolandi perform as an alien-esque insect. VFX studio JAMM realized a number of CG creations in the video, including a flying centipede and a cocoon that houses a human girl.

“We’re used to doing complex creature work, but the cocoon was a new type of challenge,” says CG Artist Zak DiMaria. “It was important to [band member and director] Ninja that the cocoon looked very real, but also unlike anything people had seen before. We based the material of the cocoon on different molds and fungi to ground it in reality. Then we simulated it and allowed it to evolve and grow on its own into a very detailed and beautiful structure.”

Then there’s Maroon 5’s promo for their single “Three Little Birds.” It begins with the band members in a seemingly traditional setting with their instruments, until each is transformed into varying – and increasingly trippy – forms. Ingenuity Studios crafted these effects by augmenting both the band’s performance done against greenscreen and utilizing in-house motion capture of five separate dancers.

“For each of the band member’s effects,” outlines Ingenuity Visual Effects Supervisor and Creative Director Grant Miller, “we first needed to track the camera, then RotoMate a character, complete with instruments, over the top of their performance. Then, in addition to the dancer mocap, we also captured a variety of handheld camera moves which were used throughout the video.”

A number of simulation effects were used during the promo, with Miller noting that the water dancer was one of the toughest. “With fluid simulations like that we’re always up against time. It was one of the first looks that we locked down, but one of the last to be completed, as every aspect is just painfully slow. The body made of cars was another challenging one. Getting the cars to pass smoothly over the body’s surface and appear to be in continuous motion required a lot of visual trickery, scaling cars away and playing things to the camera heavily.”

Ingenuity Managing Director Mike Lebensfeld says “Three Little Birds,” along with Taylor Swift’s “Blank Space” – which itself has more than 2 billion views on YouTube – stand out as some of the studio’s most memorable videos in recent times, with hugely visually interesting and creative VFX work required.

“On the surface, ‘Blank Space’ doesn’t look like a VFX-heavy video,” notes Lebensfeld. “However, it included a fully CG Shelby Cobra that we had to dent, tons of fog and atmosphere, and a few other surprises we swore never to reveal. It also represented the return of big-budget music video-making that ended up winning an Emmy award for its VR component. On the other hand, ‘Three Little Birds’ was a total greenscreen video with a ton of ‘in-yourface’ VFX that aren’t exactly invisible and represented this return to ‘big-budget Hollywood-level filmmaking’ in music videos.”

An Xsens MVN suit was used to acquire motion capture for the music video. (Image courtesy The Mill)

Playblasts of the robot dance animation. (Image courtesy of The Mill)

A final shot from “Free Yourself” featuring visual effects by The Mill. (Image courtesy of The Mill)

Motion of the Music

Among the major adopters of visual effects in music videos is the Chemical Brothers, whose recent release. “Free Yourself” required an army of CG robots with mannequin-like faces to form a rebellion and dance to freedom. The Mill came on board to generate the robots, utilizing motion capture and keyframe animation to do so.

“We knew right from the start that we would end up filling rooms with robots, which meant that we would need to have multiple motion-capture shoots in order to get all of the data we needed,” says The Mill Visual Effects Supervisor Sid Harrington-Odedra. “Due to budgetary constraints, going down the optical motion-capture route wasn’t really viable, so we spent some time looking into other options. We’d heard that since doing our last project for the Chemical Brothers, Xsens had released a more technologically advanced motion-capture suit, which relies solely on gyroscopic sensors that are fitted into a skin tight suit that the performer wears. The gyros feed back sensor data on the orientation of each body part into proprietary software from Xsens, which can then reconstruct a pose in real time, so we were able to watch performances on set, as they happened.”

Along with the capture, The Mill also keyframed final dance moves, and then also worked on ways to differentiate the three different types of robots that populate the promo: a humanoid male, a humanoid female and then a slightly less advanced faceless robot (referred to as ‘Bob’). “To try and keep things efficient,” outlines Harrington-Odedra, “we wanted to try and do as much of the variation with textures and visibility switching, so geometry-wise we kept it to just these three models, but then built in the ability to add and remove faces and the protective parts around the limbs.

“However, most of the variation came with the texturing and shading. The lead lighter, Clement Granjon, set up a shader in Houdini that would apply different levels of dirt, dust, scratches, decals, color schemes and even stickers to the robots. The chosen scheme for the robots would be laid out at the Maya end and then the attributes would be automatically picked up when rendering with HtoA.”