Last weekend in Las Vegas at Sony’s inaugural Playstation Experience event we unveiled gameplay of Uncharted 4 to a cheering crowd of thousands. It was an amazingly tactile experience as game developers usually put videos out online and can only see the reaction from comments. Hearing the crowd roar when they realised it was live and ingame was incomparable; confirmed by an accidental drop into the abyss where we’d yet to create the death animation sequence. I hope the 15 minute video below shows the lengths to which the team has pushed the fuidity of movement and overall character fidelity – animating this guy for the last three quarters of a year has been a blast!
Later on, the character team gave a presentation on the Nathan Drake model and the level of fidelity they are achieving, from physicalised hair and clothing to shaders that render mud, water and sweat. At around the 50-minute mark you’ll get a glimpse of the 800+ poses we have at our disposal when animating Drake’s face, requiring animation controls that have been as much an exercise in usability as in displaying emotion.
Below is an interview I gave a few months ago for a UK magazine on the topic of photo-realistic graphics that I have kindly been given permission to reprint. I provided an animator’s perspective as it also featured insights from developers working on racing games that are at the forefront of realistic mechanical rendering – something I hadn’t considered before. It would be interesting to know others’ thoughts on the matter, as it touches on that timeless classic, the Uncanny Valley.
From an animation perspective, how do you view the prospect of photo-realistic graphics?
I believe that years ago the game animation community realised it was something of a false god – that rather than realism we should instead be pushing for believability, (not realism), in our characters. It’s a lot easier to make a convincing performance with a character with exaggerated features when you can concentrate solely on acting instead of worrying about details such as the perfect movement of skin and muscle across a skull or the most accurate rendering of wetness in the eye. Ultimately it depends what kind of experience you’re hoping to achieve with your game. Assassin’s Creed simply wouldn’t work as a cartoony game, and the reverse is true for the Mario series.
It’s currently tapped for several game of the year awards so now is a good time to write up my notes on David Lam’s talk from the 2013 Montreal International Game Summit on the cinematic process of The Last of Us, which he also kindly gave at our studio later the same week.
David was tasked with supervising the animation for cutscenes on The Last of Us, where he said there was even more of an emphasis on story and characters than Uncharted. In particular, there was a strong focus on the relationship between the lead characters, primarily the 180 degree transformation of Joel’s attitude towards Ellie over the course of the game. While being mindful of sensitive story elements, David jumped straight in by showing everyone present the final scene of the game – (note: there are story spoilers below also). This scene, he said, best illustrated the team’s mantra of “Grounded Realism”, with a down to earth, life-like approach to performances in order to create empathy.
Here is a video on Far Cry 3’s full performance capture technology, which is virtually identical to Assassin’s Creed III’s given that we used their workflow entirely albeit from a third-person perspective. I can’t imagine recording face, body and voice separate again after seeing the subtle nuances picked up by all three working together in sync.
Marc is the Technical Director in charge of R&D at our Montreal mocap studio, (we have one in Toronto also), so oversees the motion-capture technology-sharing on all Montreal projects. For more info on FC3’s character pipeline you can see an additional talk by Character Technical Director Kieran O’Sullivan here.
While I may disagree with their approach to interactive storytelling, one can’t deny Quantic Dream’s ambition and accomplishment in terms of performance capture. Probably more than just the purported tech demo, the level of emotion captured in this piece sets a high standard with simultaneous body, facial and VO capture fast becoming the standard in our post-L.A.Noire era.
This generation’s long life-cycle has been a mixed blessing. On the one hand, so many years without new hardware has allowed us to push the limits of our current engines and learn so much about finesse and polish without the need to relearn or start from scratch. The challenge this time around has been more about real artistry in both visuals and game design without the ability to hide behind some major leap in visual fidelity or previously-impossible technology opening up new game concepts. On the other, we’re getting to a state of diminishing returns In terms of effort to rewards, and as a gamer I really want to see something that gets me excited again in a way that only realtime interactive CG can provide. And this is the first glimpse of where we’re going.
While mostly a lighting demo of Infinite Realities‘ scanned head we’ve encountered before, seeing it running in realtime on a PC at work both terrifies and exhilirates me. Once he opens his eyes and begins to move around, it will be our task to breathe life into characters that look like this in immersive worlds that are consistent and do not break the suspension of disbelief.
Since arriving in Canada, I’ve worked exclusively on games with character casts are in the hundreds, requiring procedural and systemic solutions to create their animation. Perhaps this leap in character realisation will instead force us to concentrate on stories with fewer actors that have an unprecented depth of character.
Everyone’s talking about it so it would be rude not to post that the new trailer for Rockstar’s (Sydney-based team Bondi’s) L.A. Noir dropped yesterday and the facial performances look fantastic. Using Depth Analysis’ MotionScan technology, they appear to be going all out to capture the performances of apparently over 200 characters.
While the visual benefits are obvious, I can’t help but wonder just how much it cost the production for so many actor contracts when not only voices but likenesses are required. Additionally, the production task must be one of the hardest parts of the game, considering that at the time of writing they’re still hiring senior roles for the cinematics team around 6 months before the planned release date.
That said, I’ve really been keeping an eye on this since E3 so can’t wait for spring to roll around, plus I’m a Noir nut since a Humphrey Bogart stint last summer.
Activision CEO Bobby Kotick says a lot of things. While he has talked at previous investor conferences about facial animation, (curious in itself as an investor buzz-topic), he joins a growing line of non-animators claiming to have overcome the biggest challenge in CG acting. Speaking at the recent America Merrill Lynch conference:
“This has been the Holy Grail in a lot of respects for video games – the ability to have characters on the screen that you can have an emotional connection with. The medium for the last 25 years has been very visceral, interactive, immersive medium – but it was very hard to have characters to actually have empathy towards or an emotional connection with… or that might make you laugh or make you cry; be some catalyst for an emotional reaction… Call Of Duty: Black Ops is the first game where we’ve been able to perfect the facial animation, mouth movement technology so that the lines that are being delivered are believable. The facial animation looks like a real person”
As to my position on the still theoretical Uncanny Valley, I’m convinced it more than certainly exists in Masahiro Mori’s pure sense as I’ve seen (or to put it correctly, felt) it in films like Beowulf and Midnight Express, though in games we often mistakenly attribute it to a combination of bad rigging and animation – failure not even near the Uncanny Valley.
Once again, the Japanese Softimage site has posted information on another showpiece title – and they spend a heavy amount of time talking about how Softimage interfaces with Motionbuilder. This is encouraging for me as I’ve decided to dive fully into Motionbuilder for my current project after finding it to be the most rounded solution for mocap, keyframe and facial animation out there.
When we initially showed Mass Effect at E3 2006 I recall a handful of Square developers attending to evaluate the facial animation. While it looks like their production methods are somewhat dated due to the long development cycle, playing FFXIII shows the eventual result to be outstanding – presumably due to their dedicated engine for facial closeups and meticulous planning.
Still with Capcom’s fighter, the more I play it the more I realise the actual animation is merely “functional”, but I imagine that’s what is required to ship a reboot of a franchise where every animation is subject to timing changes for game balancing throughout the project. What appeals most about this visuals are the incredibly solid models and their accompanying rigging and facial poses, so it’s nice to see that the Japanese Softimage site has a page up regarding both these aspects, (with a link to another page demonstrating Resident Evil 5’s volume-retaining arm rig too). Check it out here.
Via the Google translation I see that the game has 25 characters of around 16,000 polygons each, comprising some 5000 animations. The rigging videos are of most interest however, highlighting both their facial & finger sliders and the unique controls for Dhalsim’s squash and stretch limbs. In a break from what I’m used toÂ , the team take a less modular approach to facial expressions, with broad sliders for various facial expressions as opposed to sliders for each area of the face which can afford greater control for the animator but proves more time consuming and being prone to going off-model. This might be a viable approach with such stylised characters however, and they control the following variables: