Facial animation: A new scene to be believed

With its real cast and arty direction, 'LA Noire' looks like a movie. It's one of a number of titles pushing graphics to the limit

There's a stutter as the suspect leans forward towards a desk in an LA police station. "I owned a gun, yes, but, but..." he stumbles. His eyelids blink, his face is strained, then his confidence picks up. "It was stolen in a burglary a couple of years ago," he asserts, spilling out an explanation he feels may satisfy and persuade the officer sitting across from him.

It's 1947 in a corruption-led LA and you are detective Cole Phelps. For a while, the people who inhabit this seedy city have expressions which appear uncannily lifelike. This is a video game and you control Mr Phelps. It's your job to question victims and break down suspects in the hunt for the truth. And it is those faces which hold the clues.

LA Noire is the latest blockbuster by Rockstar Games, publisher of the Grand Theft Auto series. Drawing upon film noir, it is a startling technological accomplishment which uses cutting-edge facial animation.

The game revolves around the art of interrogation, relying on players reading the reactions and responses of the different characters, in order to separate the truth from the lies. It is why its developer, Team Bondi, wanted to ensure that facial motions were as close to real life as possible, and that, in turn, is why this game is turning so many heads.

"With previous kinds of technology like facial-motion capture, you could never really tell whether somebody was lying or not. With this technology, what you see is what you get," says Brendan McNamara, LA Noire's writer and director, referring to a technique called MotionScan.

Developed by McNamara's Sydney-based company Depth Analysis over six years, MotionScan captures the expressions of real-life actors. So in LA Noire, Mad Men's Aaron Staton does more than lend his voice to a character, as Ray Liotta did for Grand Theft Auto: Vice City. Staton acts out the role (in this case Cole Phelps). And before the actors take their seats and begin their performance in Depth Analysis's stark, white studio, they are styled and made up in a way not dissimilar to being prepared for a television or film shoot (indeed, in LA Noire, Hollywood make-up artists and stylists gave the actors a 1940s transformation).

This is captured by 32 cameras which are calibrated and aligned to record 360 degrees of the actor at a rate of 1,000 frames per second. A 3D model of each actor's head and, more importantly, every facial movement and twitch, is built so that what is seen within the game is almost indistinguishable from reality. Unlike motion capture, which many developers use, there are no dots or markers on the actors' faces, nor any uncomfortable camera headsets tracking their delivery. The result is far more impressive. What we are seeing is the progression of motion capture to something altogether much more advanced.

So far, the technology extends only to the heads of actors, but given time, full-body capture using MotionScan will further evolve the possibilities for both games and film.

"MotionScan was born out of a unique philosophy to enable developers to not only capture very photorealistic and realistic performances in a new way, but also empower them to look at how they can engage audiences differently through storytelling and interactivity," says Jennie Kong, Depth Analysis's head of communications.

"In the past, there has been a lot of need for heavy animation touch-up in post-production. With our technology, there is no need for it; on every shoot the technology works to capture everything, from their make-up, hairstyle, all the wrinkles, facial details and imperfections of the actor's skin, their mannerisms and nuances.

"More significantly, it captures the most authentic performance and adds humanity to the bits and bytes of data."

Gaming's quest for visual realism has become something of a Holy Grail for some developers and it has been since the early days of video games, when a title's worth was often based on its graphics (consider how the terrible but lush-looking Rise of the Robots in 1994 managed to earn itself 90 per cent in Computer and Video Games magazine). Today, there remains an insatiable appetite for good-looking titles, and games are increasingly becoming films that we control. While a stylistic approach to graphics is favoured by many, some feel there is merit in producing titles that appear as true to real life as possible.

Certainly, boundaries continue to be pushed. Gran Turismo 5 cost £60m to develop and it holds the record for the number of polygons per car (400,000, 100 times more than in GT4). The makers of Uncharted 3, due out this year, have spent a year performance-capturing actors for this cinematic adventure offering, and the leading football games Fifa and Pro Evolution Soccer faithfully recreate the action.

"This year, we'll be premiering a number of never-before-seen features that continue the push for authentic realism," says David Rutter, producer of Fifa 12, due out later this year. "Our new head 3D capturing system – where we take a 360-degree photographic scan of players – has led to the most believable graphical representation of footballers ever."

Gaming is not afraid to seek external help, either. Uncharted 3's 82-year-old motion-capture and voice director, Gordon Hunt, is a Hollywood veteran. But there is still some way to go. Epic Games co-founder Tim Sweeney believes we are around 10 years away from achieving photorealism in games.

One of the biggest problems in attempting to create photorealistic faces is the threat of falling prey to the "uncanny valley". The term has its roots in robotics and was coined by Masahiro Mori in 1970. He said the more human-like a robot becomes, the more people are attracted to them – but only to a point. When they appear too lifelike, people begin to see them as creepy. Much of it is due to the eyes appearing rather cold (consider the film Final Fantasy: The Spirits Within) but a deal also rests on facial movement and lip-syncing. McNamara claims LA Noire has defeated the uncanny valley.

It remains an issue for many games developers and film-makers, however, who have to make a choice between working backwards from humans using techniques such as MotionScan and motion capture or else employ a cartoon look, often exaggerated, in order to sidestep the problem.

"The spectre of the uncanny valley stalks us all," says Charles Cecil, creator of the Broken Sword adventure games, which used cartoon graphics rather than attempting photorealism. "If a developer decides to aim for photorealism, then they had better ensure that the models and animation are highly realistic and that they can afford a very substantial budget."

Cecil was also the producer of the Doctor Who Adventure Games, which were created as a supplementary extension of the television series. He again decided against trying to be too realistic. "For Doctor Who, we sought a balance between depicting the characters both realistically and slightly caricatured," he says.

A similar decision was made by the team behind the action-adventure game Enslaved: Odyssey to the West. Developer Ninja Theory's aim was to make its worlds and characters believable but not necessarily photorealistic. "We wanted gamers to be fully engaged with the narrative, but we still wanted to have the freedom to be artistic with our creations," technical art director Stuart Adcock says. "Our focus was on doing the performances of our actors justice rather than trying to replicate them identically in-game. There is little artistic merit to be had in just replicating a real face."

Ninja Theory's approach does, at least, avoid the moral issue of gaming's more realistic titles, that being if characters look too real, should gamers be asked to shoot and kill? Heavy Rain was one title which succeeded in not only exploring the choice players have within gaming environments but in encouraging a degree of affinity with the characters. Its director, David Cage, has spoken of the need for greater emotion in games and a move away from more traditional gameplay elements.

"There is no real meaning that is possible when the only thing the hero can do is shoot other people or jump on platforms," he said in March. "As a consequence, most games are emotionally limited. The kind of emotions you feel in most games are quite basic; they are about frustration, competition, anger."

LA Noire follows in Heavy Rain's footsteps to some degree, with emotional perception being as a key element to the gameplay. It shows that when games start to look real, they begin to act real, too. And it's at that point where button-bashing gives way to people skills and opens up a whole new world of play.

'LA Noire' is out on 20 May (Xbox/PS3)