LONDON — With Martin Scorsese’s The Irishman expected to battle Once Upon a Time in … Hollywood and 1917 for the best picture Oscar in February, all eyes are on the special effects team that made the sprawling crime epic possible.
The Irishman unfolds over decades, with the 76-year-old Robert De Niro and his co-stars playing their characters from their 30s into retirement age, a feat that’s made the film one of 2019’s most acclaimed movies.
It’s all possible through new digital de-aging techniques that in the past year in cinemas have shaved decades from Samuel L. Jackson’s face and turned back the clock to the 1990s for Will Smith. When Monday’s Academy Award nominations are announced, Captain Marvel and Gemini Man could see their names called along with The Irishman in the visual effects category.
Each film has arrived at its reverse aging trick through a different technique, leading some to call 2019 a monumental year for de-aging in film.
To many, The Irishman stands out from the field, thanks to its complete avoidance of tracking markers — dots painted onto actors faces that allow computers to mathematically replicate facial movements and manipulate them as the director sees fit.
The youthful transitions of The Irishman are the work of Pablo Helman, visual effects supervisor at Industrial Light and Magic, who’s an Oscar nominee for his work on Star Wars: Episode II — Attack of the Clones and War of the Worlds.
Helman says the decision to forgo tracking markers came directly from Scorsese and De Niro.
“He’s not going to wear a helmet with little cameras in there,” says Helman. “He’s going to want to be in the moment with Joe Pesci and Al Pacino on set, with no markers on him. So, if you’re going to capture the performance, how are you going to do that?”
Enter the “three-headed monster,” a unique camera rig that has a director camera in the center and two “witness” cameras on either side shooting infrared footage. That allowed Helman to eliminate shadows created by on-set lighting. The shadows could potentially interfere with the geometric facial shapes constructed by de-aging software.
“You’re not interrupting the director’s thread of thinking,” explains Helman. “You’re not changing the light on set, but the computer can see in a different spectrum.”
While the team at Industrial Light and Magic was working on The Irishman, another group of technical wizards were experimenting with de-aging at director Peter Jackson’s Weta Workshop in New Zealand, creating an entirely digital, 23-year-old version of Smith for the action movie Gemini Man.
“Since I started visual FX 25 years ago, it’s been the Holy Grail,” says Bill Westenhofer, one of the film’s VFX supervisors.
“You have that many years of expertise of looking at a human face and knowing what’s wrong. So, to try and get all the different things together and get it to pull off right, that’s been the challenge.”
To create the character of Junior — a younger clone of Smith’s assassin Henry Brogan — the superstar wore the traditional gray tracksuit, complimented by a head rig with two cameras capturing his facial expressions via tracking markers.
“We decided [on] a ground up approach to build everything from scratch — from the skull all the way to the skin pores, all the way to the animation and the final kind of oil in the eyes was really the best approach we could take,” says Stu Adcock, head of facial motion at Weta.
Before filming commenced on both Gemini Man and The Irishman, the teams at ILM and Weta drew up test footage to show the films’ directors what they were suggesting was possible.
For Gemini Man, it was a clip from the 1995 movie Bad Boys into which they inserted two shots of their new, digital Will Smith and asked Ang Lee to spot the fake.
For The Irishman, De Niro also returned to the 1990s, performing the Pink Cadillac scene from Goodfellas before being de-aged in post-production — convincing an initially skeptical Scorsese that he could bring the long-gestating project to life.
Helman and his team then spent two years looking through old movies and cataloging the targeted ages that De Niro, Pacino and Pesci would appear in The Irishman. They created a program — similar to that used to create online “deepfake” videos where one actor’s face is swapped for another’s — which would check their work on the movie was heading in the right direction, with the system “spewing out” hundreds of images for cross-referencing.