The Last of Us – Cinematic Journey

It’s currently tapped for several game of the year awards so now is a good time to write up my notes on David Lam’s talk from the 2013 Montreal International Game Summit on the cinematic process of The Last of Us, which he also kindly gave at our studio later the same week.

David was tasked with supervising the animation for cutscenes on The Last of Us, where he said there was even more of an emphasis on story and characters than Uncharted. In particular, there was a strong focus on the relationship between the lead characters, primarily the 180 degree transformation of Joel’s attitude towards Ellie over the course of the game. While being mindful of sensitive story elements, David jumped straight in by showing everyone present the final scene of the game – (note: there are story spoilers below also). This scene, he said, best illustrated the team’s mantra of “Grounded Realism”, with a down to earth, life-like approach to performances in order to create empathy.

Following Uncharted, the team wished to steer towards something more mature. A new core team, started by creative director Neil Druckman, was put in place and eventually ramped up to 175 people at the end of the 3 year production. There were around 30 animators at most, with a total of 90 minutes of cutscenes plus 2 hours of IGCs (In-Game Cinematics).

Because their games are realistic, the team relied on mocap for details & subtlety in the body though stuck with Naughty Dog’s tried-and-tested keyframe approach for facials as they didn’t have a facial mocap pipeline 3 years ago. At this point David stressed that the actors cast are very important, with the animators using their strong performances as a foundation.

Performance

In order to maintain the grounded, realistic style it was important to avoid over-the-top performances, instead aiming to show subtle emotions such as doubt, insecurity and fear. As a reference, the team used The Coen Brothers’ No Country for Old Men, where protagonists are driven by motivations like fear, greed and desperation rather than traditional video game fare. Actions, and therefore moments, are subdued yet more powerful because of this.

The animators’ challenge was not to carbon-copy the acting, but to instead capture the essence or emotion – sticking to the performances 75% of the time. Animators needed to amp-up and push performances to make them more compelling. This was made even more challenging because it was a very tightly run project under the creative director. They had to be especially careful of deviating from the characters and overall style of performance. Realism is also about exaggeration, but push too much and it’s cartoonish.

David aimed instead to interpret and “feel” poses rather than hold on them, stating that “We don’t talk pose-to-pose.” Looking for strong storytelling poses, he illustrated his point with the expression of anger on one of the characters that flashed on but didn’t remain the same. If a character is angry, they are going to stay angry until something drastic changes.

Eyes were an important factor in TLoU’s storytelling with a focus on very natural eye-lines, where looking away is when characters begin to internalise. A strong facial expression driven by eyes can show fear, doubt, loss… David placed importance on where pupils sit in the eye-socket, especially in close-ups where even the chin level invokes an attitude, and finished with the observation that blinks mark a stop.

The facial rig, (which you can see in more detail below), was designed to best recreate a muscle-driven face. It used macro controllers instead of micro in order to control muscle groups, and blend-shapes for unique features like dimples and corrections. There was a desire to show stress in the face and skin. Using Ellie’s yawn as an example, massive volume changes not possible with bones alone were apparent – where viewers should “feel” the change. Another example of Joel holding off a Clicker illustrated the taut muscles of his brow. All these details combine together to sell the performance, with lighting playing a huge role towards the latter half of project – literally revealing the characters in their best light.

Work done on the mocap mostly centered around modifying the existing performances, such as hardening the timing during combat actions to take out undesirable anticipation etc, and generally adding weight to characters. The animators would work on top of the existing action, unafraid to “butcher it” in order to recreate scenes, yet careful not to break the experience or create jarring moments.

Related to this, there were lots of conversations about acting choices throughout the project. Often asking the question, “Does this feel natural?”, the team’s approach was, “less is more”. As an example, Troy (Joel) Baker’s first take on his daughter’s death was greatly overacted so they had to dial it back. The final, powerful result showed much more believable transitions from denial to grief, then to acceptance and finally to shock. As in all of Naughty Dog’s games this generation, the overall pacing is dictated to you as a player, with the aim of hooking you then dropping you. Importantly, the team strived to give audiences moments to breathe and soak it in. As such, even the gameplay didn’t track an emotional high throughout. Relating to this, this map of  The Last of Us’ Dramatic Structure from game designer Kevin Sultan shows clear peaks and troughs of intensity.

Cinematography

The word David used was “participatory”, evoking a sense that the camera is “right there in close, lying down on the floor with the actors”. He again used the scene of Joel’s daughter’s death to show how a more roaming camera reveals moments – moving to show the reaction of Joel’s brother to the event. In addition, playing with depth via lense choices was used to convey mood. The Last of Us features some of the tightest shots I’ve yet seen in a videogame, which were often used to make characters look alone. Before going to a shoot, a lot of pre-vis camera work was done beforehand.

Cohesive Production

Unlike Uncharted, this project was tightly driven by the creative and game directors. Daily reviews were held for the animations, with David himself having a weekly one-on-one with the directors. However,  much of the methods and technology were adapted from the legacy of Uncharted with an aim to push the tech and workflow. The team would place emphasis on talking things through in order to “work smart”.

While only 6 animators worked in-house on cutscenes, Sony San Diego was drafted to increase the numbers. Cinematics pulled ahead of the rest of the game early on in production by “front-loading” story elements they were confident would be locked down, accepting that around 30% was going to change. That said, they were mindful of staying close to game, not going too far ahead lest they limit the design team and their choices.

Working cohesively with other areas of the development team brought together the gameplay, in-game cinematics and cutscenes so that the game was tight and cohesive despite involving different disciplines and workflows. This invovled a lot of teamwork across disciplines, (designers, animators, artists, lighters, etc), and this successful approach can be seen in the final result.