[Last Guardian Director, Fumito] Ueda-san ended his talk saying that good animators should not only be discovered, but nurtured and educated. He thinks that animators are incredibly valuable to any game project. “I think great animation is one of the most important advantages the Japanese entertainment industry has over other regions.”

The best teams I’ve been involved in have had an air of education about them, and the best studios feel like universities where everyone can learn from those around. The right balance of knowledgeable seniority and youthful enthusiasm raises the quality bar across the board.

Find the original (Japanese) write-up here. [via Team Ico Gamers]

That’s more than can be said for my human companions. Our drunk driver Roger didn’t talk much, though he was always friendly, and the ice-queen with whom I shared a tent was 100% business. I even hung a picture of her on the wall and had the car re-painted in her favourite beige, but all to no effect. She wouldn’t even look at me…

Looks like a simple rig in Maya to compliment the game nicely, and they use morph shapes for the facial emotes which seems a natural fit given the squashiness of the character.

Your partner at Quantic Dream, Guillaume de Fondaumière, has said that this is the biggest motion-capture project attempted in a game to date – what are the implications of this from a development perspective?

We bought our in-house mo-cap system in 2000 and we’ve had a full-time team working with it since then. We developed proprietary technologies, tools and pipelines to produce high-quality data in a very limited time frame. Heavy Rain was more than 170 days of shooting with more than 70 actors and stuntmen, plus 60 days and 50 actors for facial animations. We recreated most props on the set to allow actors to know what’s around them and to have the right contacts with their environments.

[via Edge Magazine]

Click here to view the embedded video.

Click here to view the embedded video.

It was becoming painfully obvious that my old PC just wasn’t cutting it for a supposed game developer, (I’ve never really been a PC gamer), especially when it wasn’t even hitting the minimum requirements for some of even the most basic of software. I was, however, loath to give up the dual-screen setup I’ve grown accustomed to at work and at home so have found some solace in the FW’s unique 16.4” widescreen.

If it even allows two documents or webpages side by side during my various forays into graphic design or simply gives the 3D viewports a little more breathing space, it’ll be worth finally breaking free of the constraints of the desk. Now to find a café sur le Plateau…

Sony Pictures Imageworks: A Believable Character Postmortem: Motion Capture on the Virtual Set of BEOWULF

Parag Halvadar – Lead R&D Engineer

Hailing from the same studio that created Monster House, Halvadar’s talk concentrated on facial motion as that’s a recent topic for games industry. As is often the case with movie industry approaches they couldn’t directly be recreated for use in a game development situation, but nonetheless provided an interesting insight into some of the lengths that must be gone to in search of the (some say, false) holy grail of truly photo-real virtual characters.

The first portion of the talk involved simply tallying the vast amounts of data, equipment and effort used in the production:

• 260 Vicon MX40 cameras were used synchronously to record motion.
• Body, facial and hand motion were captured simultaneously.
• An Electro Oculograph (EOG) was used to record eye-tracking.
• 20 actors could be captured simultaneously.
• Actions were captured in a 55x55x25ft volume.
• 81 actors were tracked over the course of the movie.
• 4 horses.
• 1 pony. (Only one?!)
• 46 days of shooting.
• 250 props made and captured.

The second portion detailed the methods required to bring the faces to “life”. It must be said that, despite often firmly entrenched in the Uncanny Valley as is always the case with attempts to simulate realistic facial motion, Beowulf has done the best job yet at providing real glimpses of coming up the other side. The tallies continue:

• 4 layers of face rigs.
• 3D facial models did not match actors faces in a 1:1 ratio, (Ray Winstone in particular), causing lots of marker-swapping.
• Adhered to FACS (Facial Action Coding System), to recreate all the muscles of a human face, totalling around 60 facial expressions (including head motions), with 16 different phoneme shapes.
• Face poses were created from combinations of weights of a smaller set of basic poses.
• Motion-capture values were run through a script to find the closest match with the facial expressions and were replaced with blendshapes.
• The EOG recorded horizontal and vertical eye movement, saccades and blinks via and eyepack on back with electrodes by the eyes to detect eye-muscle movements.

It should be noted that, while in my opinion the face of Angelina Jolie was the most successful and consistent in quality throughout all shots involved, Halvadar explained that hers was the scan that was deliberately adjusted the most to form an exaggerated impression of how we picture her, backing up my belief that realism is simply just not realistic enough when it comes to artistic endeavours such as this. This was also apparent in the scene of her naked, gold-dripping body emerging from the water – something which he felt the need to show several times over and was also manipulated drastically due to her pregnancy at the time of shooting.

In closing, it was most interesting of all that Halvadar’s decision to show each scene step-by-step revealed that every shot only achieved the final visual quality after a final pass was made by an animator working with video reference of the original scene, begging the questions as to why go to the bother of all the technicality when that process could be done from scratch with presumably similar results.

If absolute realism in games still is your thing, then you may wish to investigate the work of George Borshukov at EA and his Universal Capture (UCap) method. Proven in The Matrix trilogy and Tiger Woods tech demos this really is something to watch, especially since its optimisation for real-time implementation.

As an extra little teaser, there will soon be something new coming to Game Anim of interest to videogame animators everywhere over the next few weeks. So on with the notes…

Naughty Dog: Uncharted Animation – An In-depth Look at the Character Animation Workflow and Pipeline

Jeremy Lai-Yates & Judd Simantov – In-Game Animation Lead & Lead Character TD

After a fantastic opening to GDC with Ken Levine’s inciteful speech on Storytelling in Bioshock, this, my second lecture, turned out to be not quite all that I’d hoped for. I was really expecting to gain insight into their facial animation setup and workflow as my time spent with Drake’s Fortune have proven the cinematics to be something quite special and well-produced. However, the talk focused squarely on their mocap workflow which was a fairly standard 3-skeleton setup. 1 animation, 1 game, and 1 mocap – snapping poses and animations between them.

What confused those present most was, on deciding against employing Motionbuilder as part of their process due to their exporter being Maya-based, they proceeded to manually recreate many mocap-related features Motionbuilder provides inside Maya, (though with the notable absence of layers), rather than simply recreate their exporter inside Motionbuilder. This was reflected upon at the end with the closing statement – “We had a tendency to over-think things”.

My notes:

Early Considerations

• Naughty Dog are traditionally known for cartoony games. A move from a stylised look to realism would necessitate a review of their approach. As such, they decided on integrating mocap from the start, employing the service of House of Moves.
• The initial goals were to create a stable rig and pipeline. Previous games left cinematic and game skeletons incompatible, but for Drake’s Fortune the aim was to have a single skeleton shared across both disciplines.
• Early on, they decided against Motionbuilder as their exporter was Maya only and they didn’t have the skillset to allow them to recreate their exporter inside Motionbuilder. As a result the entire workflow was created inside Maya.

Tools

• They created a custom tool to reference characters. Maya already supports referencing but they improved upon this to allow character selection and easy replacement. They needed consistent namespaces for references, and so the editor was employed to better handle this.
• Their importer/exporter ran at around 300 frames per second.
• The mocap and animation rig existed in the same file. The default setting for copying mocap onto the animation rig was 1 key every three frames. One assumes this approach was taken in the absence of animation layers.
• They created an animation library, (affectionately named iToons), for easy pasting of poses, (body or hand), and pre-existing animations as well as a time-warp curve for easy adjustment of an entire animation’s timing.
• A layer manager was devised to allow for subdivisions in Maya’s pre-existing visual layers.
• One interesting addition was a visual arc tool, much like Max’s (or Motionbuilder’s) trajectory display, that would show the trajectory of a point over the course of the animation, only the keyframed positions were colour-coded and could be manually edited to affect the actual trajectory of the animated element it represented.
• Of note, their FK rig was set up in a manner that allowed it to be be manipulated as if it were IK.

The Road To Realism

• Their initial treatment of the character movement style was very agile and versatile, involving flips etc, but this was scaled back in favour of a more “everyman” approach to the main protagonist. They naturally found this difficult to mocap due to the limitations of real-world physics on an actor. To overcome this the keyframed over the mocap to afford much more exaggeration.
• The entire project ended up being around 40% mocap vs 60% keyframe. Motion was captured at 120fps, a high fidelity, but they noticed a difference between the mocap and keyframes.
• In order to smooth the discrepancy, they willfully lowered the mocap quality by reducing keyframes, sometimes to only the key poses from an action. In addition, they employed what they called “Poor Man’s mocap” – video reference, (does anyone still work without video reference?), accentuating poses, adding “punch” to timing, snapping extreme poses and removing unnecessary pauses.

The closing section of the speech was the most interesting, focusing not on the process, but on what they did with the animations after they’d been implemented in the game engine.

First up were what they called “Partials”, essentially sets of bones that could be animated independently of the rest of the body. We’ve been seeing this for years, but when combined with the rest of the multiple animations played on the character the weapon-reloads really looked natural and flowed as Drake moved from one state to the next.

The second take-away idea was their imaginative use of additive animation, where they changed not the additive loop, but the underlying pose. With around 150 different poses for the cover state, they would change the player character’s posture based on various influences such as health, state-of-duress, as well as randomness.

Looking at the image below, they at least animate using Lightwave, though how much of their pipeline relies on it is unknown. Perhaps the Japanese industry is different as a whole, as in the West we mostly use either Maya or 3D Studio Max. Lightwave is more favoured among high-end artists for it’s renderer, not its animation system.

The rig below includes the wireframe mesh of the shadow-casting model, but otherwise displays fairly standard IK/FK limbs, with an aim constraint to animate the head, (presumably also able to be turned off). This aim-target is of interest if it is actually exported with the animation and used in the real-time target system as is the case in HalfLife 2′s “Source” engine.

I like the cute 3D hands on the floor that presumably denote all the finger controls, but still prefer to have those controls seperate in a Channel Box (Maya) or Modifier Panel (Max). Note the boneless cloth garment, showing that they have a seperate system for that element rather than being bone-driven.

What really intrigues me though are the long bones at the extremities – possibly something to do with their real-time IK system, but more probably with the standout feature – the deforming collision of the Colossi themselves, whereby they calculate the player’s position on the collision as a single spherical point making it easier to react to the deforming mesh, (below).

The aforementioned real-time IK system appears to support a fairly standard 2-bone setup (affecting just the last two in the horse’s leg), save that it not only affects the limbs but also raises and angles the whole character in one solution. In past games I’ve encountered these as two seperate systems, with the latter affected by a check on the ground normal rather than an average of the various IK limbs. See below for a comparison – first without, then with the root being affected.

Finally, the most impressive (and hopefully soon to be widely adopted) element of the entire movement system centres around what they call their “motion-addition” system – essentially the combining of procedural and animator-created motion to give true fluidity without the sloppiness of purely-procedural movement, or rigidity of keyframes alone.

A physical simulation drives animation of the bones, causing the character to rotate around his single handhold, and presumably to arc the body and delay the limbs based on that motion as seen in the image below. This was not only used to afford the flailing motion of the player character while the Colossus violently attempts to shake him off, but also for the natural movement of the horse.

This combination of procedural calculation that drives motion tweaked by an animator, (or even better, purely procedural systems that are simply fed data in the form of exported animations) is the next step in more natural-feeling motion in our in-game characters.

“Movement becomes wooden if purely controlled by a program, but also a pre-made animation won’t fit in right. It is the combination which makes it work. When I saw this working in real time, I was more than a little impressed! I think it has become a truly wonderful system to combine the animator and programmer.” - Fumito Ueda, Director

For the original article in its entirety (including images), go here, with an English translation here.