“Kill Bill: Volume 3”? Now We Have an AI That Can Mimic Iconic Film Directors

READ MORE: Cine-AI: Generating Video Game Cutscenes in the Style of Human Directors (Cornell University)

An AI that mimics the directorial style of Quentin Tarantino or literally any other auteur you care to think of? The possibility is tantalizing and within reach.

A group of researchers out of Aalto University in Finland have devised a tool that generates video in the style of specific directors.

It’s so good that, when put to the test, audiences can tell which director’s style is being mimicked.

Cine-AI is in fact targeted at the automated creation of cutscenes in video games. However, its applicability to generating cine-literate visual storytelling that emulates the film language of famous directors —alive or dead — is clear.

It would also only be a hop, skip and a jump to apply the same process to auto-generate synthetic cinematography as if lensed by Vittorio Storaro or Roger Deakins. Anathema as it sounds, this will be possible at a convincingly photoreal level sooner than we think.

The particular problem that Inan Evin, Perttu Hämäläinen and Christian Guckelsberger sought to crack is laid out in their white paper published in August: “Cine-AI: Generating Video Game Cutscenes in the Style of Human Directors.”

In-game cutscenes are non-interactive sequences in a video game that pause and break up gameplay. In high-quality, AAA productions especially, cutscenes feature elaborate character animations, complex scene composition and extensive cinematography for which games developers may need to hire dedicated directors, cinematographers, and entire movie productions teams.

“Cutscenes form an integral part of many video games, but their creation is costly, time-consuming, and requires skills that many game developers lack,” the researchers explain. “While AI tools have been used to semi-automate cutscene production, the results, typically lack the internal consistency and uniformity in style that is characteristic of professional human directors.

“We aim to realize procedural cinematography, focusing on how camera placement, shot continuity and composition can be brought together by algorithmic means.”

To that end, they have devised Cine-AI — an open-source procedural cinematography toolset capable of generating in-game cutscenes “in the style of eminent human directors.”

Implemented in the game engine Unity, Cine-AI features a novel timeline and storyboard interface for design-time manipulation, combined with runtime cinematography automation.

Besides Tarantino, they also chose to train their AI on the films of Guy Ritchie (Lock, Stock and Two Smoking Barrels, Snatch, Revolver, Sherlock Holmes), explaining that not only did both directors have recognizable and unique shooting styles, but they were well known to a wider audience. That was important when it came to assessing the results.

Arguably, Guy Ritchie’s kinetic style works well for action-based video games while Tarantino’s works best for more sedate dialogue-heavy scenes.

As the basis for the dataset, they extracted 80 one-minute clips from the most highly rated movies of each director on IMDB. For added representativity, half of the clips chosen were action-heavy, and the other half to be strong on dialogue. Each clip was also assessed in terms of its dramatization level and the scene’s pace, encoded, for example, on a scale with high values given to a scene that unfolds quickly.

Importantly, these cutscenes are not baked into the game but designed to playback dynamically based on the actual game state, “rendering the production of different, static cutscenes for each possible gameplay outcome obsolete.”

With the finished clips they arranged for viewers to take the Pepsi challenge and judge which was in the style of Ritchie and which a Tarantino. Eighty percent of the responses were correct.

The team is aware of their system’s shortcomings and its potential. They note, for instance, that not all directors solely focus on camera work to express their style.

“Michael Bay, for example, notoriously employs many post-processing effects such as lens flares and god rays.”

Future enhancements to the software might introduce additional cinematography techniques including post-processing effects.

They would like to open Cine-AI up to more genres outside of action and to account for less iconic directorial styles.

“Both directors are moreover white and western men, and future efforts in extending the dataset should focus on increasing director diversity,” Evin, Hämäläinen and Guckelsberger noted. They added they were intrigued to see how well Cine-AI might reproduce the style of horror and action director Timo Tjahjanto.

Other directors cited as providing “a worthwhile challenge to procedural cinematography,” include Spike Lee, who “focuses on color and race relations and is well known for his frequent use of dolly shots to let characters ‘float’ through their surroundings.” French director Agnes Varda is also mentioned, “praised for her unique style of using the camera ‘as a pen.’ ”

The researchers don’t expect Cine-AI to “completely replace” actual human directors in video game creation but, like other developers of AI tools, see it as a co-creative in the process.

“The cinematography requirements of AAA games tend to get extremely sophisticated in terms of style, scene duration and size,” the paper explains. “While AAA companies will likely continue relying on dedicated production teams to achieve the desired level of cinematographic quality, they can utilize Cine-AI to create prototypes for their cinematography design, automatically generate shots to inspire new ideas, or use the storyboard feature to quickly iterate on possible shots, following a specific directorial style.”

Both the proof-of-concept dataset and the source code are now publicly available under an open-source license. The Finnish team is inviting other researchers, game developers and film enthusiasts to join them in taking the project further.