This Canadian VFX studio is using AI to cosmetically alter your favourite actors

TORONTO – Wondering how some stars seemingly never age? A Toronto VFX company says the reply is in a growing post-production trend by which cosmetic touch-ups are routinely made on actors’ skin and hairlines using technology.

Matt Panousis, partner and chief operating officer of the visual effects and tech studio Monsters Aliens Robots Zombies (MARZ), says the industry has been shifting within the direction of subtle, virtualcosmetic alterations for a while.

“What you discover as you go deeper is quite a lot of these celebrities are brands,” says Panousis. “Like a brand that should remain consistent — so does a star’s face.”

The studio, which began as a startup in 2019 and does various visual effects work including bringing a disembodied hand often called “The Thing” to life in Netflix’s “Wednesday,” says the cosmetic side of VFX, whether fixing imperfections or de-aging actors for flashback scenes,is a big a part of their business.

A couple of 12 months and a half ago, they began using software called Vanity AI — a project that took the corporate three years to construct and which they now plan to license to others — for work resembling removing pimples, crow’s feet, and under-eye bags.

In a single example, they helped smooth out laugh lines and fade brow wrinkles on Vancouver-born actor Joshua Jackson for a flashback sequence within the Peacock series “Dr. Death,“ by which he portrays American neurosurgeon Christopher Duntsch. Within the scene, the doctor is shown as a football hopeful during his university days,which takes place several years prior to his infamouscareer in medicine.

In one other, an otherwise visible wig line is made invisible on actress Kathryn Newton, who dons a brunette hairpiece to portray Cassie Lang, the daughter of Avenger Scott Lang who goes on to turn into a superhero herself on this weekend’s “Ant-Man and the Wasp: Quantumania.”

“There are storyline touch-ups after which there’s the purely cosmetic, so after we say 80 to 85% of all productions use visual effects, a giant chunk of that’s just the cosmetic stuff,” says Panousis. “Give it some thought, why do people like Snapchat filters? Celebrities are people too.“

Removing shine, blemishes, and de-aging is among the many cosmetic work that has grown in demand, says Panousis — and the expectation is there even for TV productions, which generally work on tighter timelines and budgets.

“These guys have a fraction of the budget and a fraction of the time that their film counterparts have,” he says.

“It’s only gotten worse and with the proliferation of streaming and the quantity of content coming out. There should not enough VFX artists on the earth to do the work.”

What makes Vanity AI faster, Panousis says, is the way in which the tool can predict where the perfect placement of cosmetic fixes needs to be moderately than continually modifying a face frame by frame.

Through a series of dials, users could make alternationsthat could be applied to multiple frames until it’s a moving, fully altered shot — a process that might previously take days, in comparison with mere hours with their app,in accordance with MARZ.

Panousis adds that because the demands for cosmetic changes and time-jumping storylines turn into more popular, the celebrity who fancies a more touched-up look is now an industry expectation they’ve also had to satisfy, albeit a soft-spoken one.

“Sometimes it’s contracted, which is often on your major A-lister celebrities who’re the intense cases,” says Panousis who declined to call names because of contractual obligations.

”They’ll have it built into the contracts that say, unless there’s a certain budget related to some form of cosmetic enhancement, they won’t do the project.”

“A lot of the industry’s work is entirely invisible and Vanity AI as an application, is supposed to be entirely invisible — you’re not likely speculated to know if an individual has done some work in the event that they’ve done an excellent job,” says Panousis.

“It’s great for an actor who had a breakout of pimples, or one other who experienced a foul hangover, whatever areas you choose, you’ll be able to clean it very simply.”

Panousis acknowledges that in an era of synthetic media like deepfakes, there are other AI solutions to creating cosmetic changes. Competitors exist in various degrees, one among the more notable being Disney’s FRAN by Disney Research Studios which allows users to re-age and de-age someone at little cost.

Still, he says Vanity AI meets Hollywood’s high-quality expectations given its use on multi-million dollar projects resembling “Spider-Man: No Way Home,“ and ”Stranger Things 4.“

This 12 months will mark the primary time MARZ will make the capabilities of Vanity AI available to large and small production studios all over the world to “democratize the consequences.”

“Democratization means bringing down the barriers to entry and making the work accessible,” says Panousis.

“When you give it some thought, quality VFX is something that’s only ever been available to the most important and best productions, but when we will price it properly for these different segments in a way that’s digestible, we consider that the tool is completely going to be utilized in other segments of entertainment.”

Panousis ultimately envisions Vanity AI eventually being reasonably priced for the typical indie creator as well.

The goal for MARZ is to license out the app, which Panousis acknowledges may mean giving others the flexibility to perform the identical work they do.

“We’ve got to get the tech into others’ hands and that would thoroughly be our competitors and that could possibly be other VFX corporations,” he says.

This report by The Canadian Press was first published Feb. 17, 2023.