While what we’re doing is very segmented – the final look comes in the final moments, and you get impatient – in building a videogame, it’s more of a gradual thing. In games, you’re layering it consistently; so by the time you’ve got your engine running really well, even for the person who’s working on low-level stuff, everything looks great.

And on this side, we’re still trapped in this universe that requires such things as massive rendering times, which makes it impossible for us. The better it looks, the heavier it is; for us to change the movement of this character, we have to backtrack and change all these other elements. Whereas on the game, they could look at something on the screen – a door handle, say – and change that doorknob to make it look beautiful. For us, it gets heavier the better it looks. There was a moment I was watching what they were doing and I thought, “I want to do videogames!” Because they’re building this thing – and once it’s in working form, they’re working in real time on beautiful stuff, all within the engine.

One of my most enjoyable parts of game development is the pre-visualisation stage preceeding pre-production. It’s when your imagination can run wild before the harsh realities of real development take hold. I find the animator in a strong position here as complete sequences and gameplay scenarios can be mocked up without any need for a programmer.

Industrial Light & Magic: Building Iron Man

Marc Chu – Animation Supervisor

Beginnng with his history, Marc joined ILM in 1994 and has since then served on 20 films, perhaps most notably as Animation Lead on “Pirates of The Caribbean” character Davy Jones. As Ironman was Marvel’s own first fully self-financed feature, they had 6 different companies competing for VFX work. It was interesting to see that despite its reputation, ILM must still compete for bang-for-buck value as film studios are keen to shop around. It comes as no surprise though, that the work was won in part on the back of the impressive Transformers work.

To this end, he showed a rough animation test of Iron Man taking off done over the course of two weeks. ILM has plentiful archive footage from which it can draw resources, and for this piece air footage repurposed from Ang Lee’s Hulk was used to create a high-quality flight sequence.

The Suit Model

The talk continued with Marc taking us through the various versions of the Iron Man suit, which were farmed out to seperate VFX houses. The Mk I was handled by Vancouver studio The Embassy, with the Mk II (above) by The Orphanage. Stan-Winston’s Studio was drafter to create the full non-CG suit, with the traditional approach of working on small maquettes to reduce costs.

From this base, ILM built the fully working Mk III suit in Maya, showing tests to demonstrate difficult rigging areas such as around the pelvic area. 44000 photos and 1100 quicktime movies were gathered for reference, and interestingly Marc displayed Photos as frames in .mov files for ease of access. These photos contained everything possibly having to be referenced or recreated in CG.

As part of the CG suit creation process, the team would make daily turntable renders to compare to the real Winston models and compared shaders to real iron balls. Regarding this, around 6 different damage materials needed to be created for the suit at various stages throughout the movie.

Chu admitted that Iron Man’s flight was based on Star Wars Snowspeeder movement, though initial air-brake test concepts were weak – based on an idea of the hero being some kind of “Samurai In The Sky”. Of interest, many of the initial movement tests were done by the suit modeller, whom one assumes must have been rather multi-talented, especially after we were shown the depth to which Iron Man’s endoskeleton was fully detailed and modelled.

Some Mk III suit modelling stats:

• 1,900 Individual Objects
• 872,000 Polygons
• 3756 Texture Maps
• 483 Rigging Nodes

The Animation

The film featured lots of “Digital-Double” shots, same for cars & jets etc. ILM made continued flight tests on their choices of air shots, with Chu admitting that the relationship between the VFX team and the Director had never been as highly collaborative as this before, with the Director asking animators to provide ideas for shots. Sometimes entire shots were created to match ILM test renders.

For the final battle with Ironmonger, ILM had to redesign the back-piston rig to avoid crashing when he moved, with Marc demonstrating some excellent keyframed walk-cycle tests. On the set, actors interacting with Ironmonger traked a low-tech “head-on-a-stick” for eyeline reference. At this point he mentioned it was more difficult working on live-action plates due to the numerous limitations such as predefned locations where the character’s head must be placed.

Perhaps the technological highlight of the talk was the section on ILM’s proprietary iMOCAP technology that allows the motion-retargetting of CG assets using just the filmed plates, avoiding the requirement for motion-capture suits. This method was used primarily to allow actor Robert Downey Jr. to have the bulky MKIII suit added back on in CG in some instances where the real-life suit would have proven too resticting.

Surprising for a humanoid super-hero, only 4 shots in entire movie used real mocap. With the tally of CG shots as thus:

• 409 CG shots, of which…
• 289 were 3D animation, comprising…
  • 254 Keyframe Animation
  • 29 iMOCAP
  • 4 Mocap

Director Jon Favreau’s summation of the hero’s movement was, “Superman takes off fast and lands slow. Ironman takes off slow and lands fast”. With this in mind, ILM had to show a visible improvement to character Tony Starks attempts at taking off throughout the movie. We were shown jet-blast tests done over a still photo, (to create an easy looping sequence), with Chu remarking that this area proved a challenge compositing all the flare VFX.

As I’d been previously informed at Giant Studios, the team mocapped an actor in a wind tunnel with his feet tied together to get an idea of how Iron Man might perform during flight, but in the end this proved useful as good reference only. Of note, the actions during air flight used FK animation only – a change from the typically IK-based rigs ILM use in Maya.

The last part of the talk focused on the machine Stark used to build and don the suit, the “Suit Machine”. This effect required a tremendous amount of work to create, and was notable mostly for the technique of recreating the background set digitally. By capturing many photos, taken in multiple exposures, ILM created a single 360 degree shot of the set – placed on sphere like a videogame’s skybox. All shots during this sequence used this virtual background, with the Suit Machine (and Iron Man) rigged by another multi-talented indivdual, a Transformers animator that did some of the most complicated shots of that movie.

Suit Machine modelling/rigging numbers:

• 15461 Objects
• 3,980,000 Polygons
• 1825 Texture maps
• 2284 Rigging nodes

In summary, the project totalled 400 VFX shots over the course of 3 or 4 months, though when pressed at the end on a shot quota at ILM, Chu explained that there is no quota in the traditional sense, and the studio never rushes animators, taking time to get each shot right. As such, one presumes they have banks of animators waiting to pick up the slack when schedules slip, as he described the practice of bringing on extra people before deadlines etc. Overall, Iron Man had 15-20 animators over the course of production.

I’ve always thought Mass retained something of the same look as all the other games rendered in the Unreal 3 Engine, but perhaps we did manage to put our own stamp on it with little tricks like the grain filter and custom depth-of-field, (the DOF in the image above was Unreal 3′s default at the time). And if I wouldn’t love to make a game locked at 24fps with motion blur.

As far as I’m aware, this is the first time they’ve run a cover story on videogame VFX, which is usually the territory of film and television only. Definitely a great step forward and some great recognition for the excellent work done by Shareef and the rest of the VFX team, even if they did kill the framerate throughout (and to a degree, post) production ;-)

Go buy them and laugh at their funny French accents.

Click here to view the embedded video.

I recently saw Beowulf in full Imax 3D and came out wondering two things.

• Why did they bother to make it in CG anyway?
• Why aren’t we pursuing full 3D in games?

I’m sure we’ve all played a round of Quake with cheap VR glasses at some point in the past, but could you imagine how impressive the current generation of games would look, even if the quality was likened to split-screen to provide two displays? The VR fad all but died outside of academia since it’s brief flirt with the mass audience via more up-market arcades in the 90s, but I’ve always believed it will resurface at some point when the technology becomes affordable, (and side-effects like headaches and nausea are overcome).

With major hollywood directors like Steven Spielberg, James Cameron and Peter Jackson now looking towards 3D to bring crowds back into the cinemas, I’m confident the experience provided by film’s linear and passive nature would be easily trounced by that of a fully interactive 3D world, the likes of which videogames have been creating for decades.

Industrial Light & Magic: VFX Used On Transformers

Todd Vaziri – VFX Sequence Supervisor for Transformers

One of the most entertaining presentations of the week, due to both Todd’s upbeat yet humble attitude and the sheer multitude of videos displayed during the presentations, ranging from multiple render-passes highlighting the various explorations of lighting and materials on the robotic protagonists to behind-the-scene shots of the film plates throughout the various layers of post-production layering.

Incredibly heartening were the animation renders illustrating the sheer amount of cheating going on when characters went off-screen. With the original brief requiring 14 robots in total, they scoped for only 14 transformation animations, but ended up creating over 140 due to each transformation being created specifically to sell the particular shot. Some examples shown had Transformers’ legs going through the ground, various parts scaling into the body to be hidden away, even bits flying off only to return just at the moment they were required on camera – just like our cutscenes!

Some animation-related notes:

• Despite having a very humanoid rig underneath all the additional moving parts, Mocap was used for subtle idling movements for the Transformers – everything else was keyframed.
• Similarly, procedurally-generated secondary (i.e. physics-based) animation was used rather liberally on the characters. The demo illustrating this (showing Optimus Prime) had only a few flaps around the shoulder and arm areas, leaving much of this type of work to the animators themselves.
• Character animation was done in two passes, with one team providing the finished body animation of the Transformers and a second group later going in to animate the vast (see: tens of thousands) number of smaller moving parts required to raise the characters to big-screen fidelity.
• Todd’s main inspiration came not from other movies based on their action and visual effects, but rather on the actual type of film used when shooting the live-action plates which results in certain lighting artifacts due to the stretching of the film image to fit the screen. In his words, “You don’t want to use VFX films as your VFX reference”.
• He had a little advice for camera-shake, being that it should always be implemented slightly after the event causing it, presumably varying by distance from the event. This is something we could easily experiment with in future games.
• Surprisingly, several of the VFX shots were created almost entirely using 2D in After Effects, with one example shown involving explosions made entirely of video footage being translated in 3D space – good old sprites are still useful when it comes to creating smoke and explosions even for the big guys – it is fair to say that Mr Vaziri is an advocate of low-fi techniques.

I’ve been to a few ILM lectures now, and at each one I’m staggered by the amount of time it can sometimes take to render just one frame for a CGI sequence. For those who still wonder how long it will take for videogames to achieve photorealism, (a fool’s quest at best), consider this gulf:

• Time currently allocated to render one frame in most XBOX360/PS3 games – 1/30th or 1/60th of a second.
• Time currently required to render one ILM VFX-heavy frame for film – anywhere from 10 to 30 hours.

Halon: The Value of Pre-Visualisation

Dan Gregoire – Pre-viz supervisor on Star Wars Episodes II and III, War of The Worlds, X-Men 3 and Ghost Rider

This talk was especially of interest to me as we were just finishing the pre-vis work for a handful of outlandish actions and animation systems in my current project – a valuable process that not only helped prove their viability, but also how the visual look of the systems will play out.

These notes will be of interest to anyone currently planning out cutscene requirements, or teams looking to pre-visualise how certain gameplay elements or level action sequences might play out in something more advanced than simple documentation or even storyboard form.

With a background in videogames and animated television, Dan was drafted in to create pre-visualisations for effects-heavy scenes midway Star Wars Episode II, then continued this work on the Episode III, choreographing difficult sequences such as the fight between Yoda and Palpatine.

In addition to working on movie pre-viz, Halon is also involved in animated movies such as the upcoming Avatar and Speed Racer, as well as the recent Halo 3 “Believe” adverts. Near the beginning of the talk Dan asked the audience “Who uses pre-viz?”, and was met with a resoundingly lacking show of hands, to which he replied, “You should be”.

Halon’s work process is essentially as follows:

• The pre-viz team can be brought onto a project long before every other element to work with the director. This is usually done at their own studio, with the director “moving in” over the period of a couple of weeks. Interestingly, during the War of The Worlds portion of the talk, when pressed about the Director’s freedom being constrained by pre-visualised shots created by Halon, he offered that Spielberg himself is familiar enough with Maya to be able to take Dan’s shots and move a camera around to explore with.
• For less tech-savvy directors, the workflow essentially revolves around the director directing Dan over his shoulder to create the type of shots required. Importantly, this is a process of exploration as opposed to the director already having the shot in his head and working towards recreating it. He also referred to it as “problem solving”.
• Sometimes Halon is even allowed free reign to create the shots alone then submit for approval, though naturally this will involve more back and forward as anyone who’s worked with outsourcing will attest.
• Additionally, their team can be brought in on-site for shots during production as their studio requires nothing more than a few laptops and Maya. As such, some in-production shots are farmed out to Halon members off-site that can work remotely when under pressure.

Software/source material used:

• Maya: Easy portability, as well as easy hiring from within the film VFX industry.
• Google Earth: Used for location scouting, gathering texture, terrain, GPS and Building data for accurate real-world environment layout. One location we saw featured an accurate recreation of a pier to exact scale.
• Sketch Up: Simple building creation.
• Face Gen: Fast facial model creation.
• Blue prints: Accurate interior layout.
• Location Photographs: Presumably texture info.
• Stock models: Used in addition to in-house creations to reduce workload for elements such as vehicles etc.

Looking at a selection of demo movies displayed, I was able to garner the following:

• Sometimes the pre-viz mock-ups used models that were accurate enough to easily distinguish the actor they were representing, with full modeling and texturing. Others involved only roughly put-together texture-less models with default smooth skinning applied, (some moved with the character’s names as polygon-text – a nice touch). War of The Worlds on the other hand involved fully finished recreations of the tripod walkers, giving away that certain pre-viz efforts were done at a later stage on that project.
• This level of detail is apparently at the discretion of the director, and in Dan’s words, “It doesn’t matter if it looks like crap”. VFX such as particles and explosions are also added at this stage, giving a good reference when handing over to ILM for full treatment.
• Similarly, the animation varied from held poses to minimal-frame animations as characters moved throughout the scene, though never in T-Pose – likely to provide for more accurate framing and composition. Nevertheless, the shots from the upcoming Spiderwick Chronicles involved fully blocked-out body and facial animation.
• Shots displayed for Ghost Rider not only composed of dynamic sequences but also some slower character-acting shots. It was later revealed that some directors wish to use pre-viz to essentially mockup most of the movie for budgeting estimates such as VFX shots required, number of actors and locations, scheduling etc. One example given was 80%-90% of total shots done, delivered in quicktime format.
• Importantly, pre-viz focuses on the development of contiguous sequences, rather than individual shots.
• Of note, Dan stressed that his Maya camera rigs stayed with real-life scenarios as much as possible, due to the nature of most of the shots requiring recreation in the real word. As such, he went to pains during a demo to avoid simply parenting a camera to a speeding vehicle, instead to an imaginary second vehicle from which the shooting would take place in real life. The best example of which was during the Halo 3 advert. The 36x26ft battlefield museum exhibit was built at a 1:1 scale, so he pre-visualised all his shots via a camera attached to a “crane & snorkel cam” in Maya, a virtual recreation of the device to be used on set.
• Apparently, “Motion Control” (the process of virtual camera movements being recorded and played back on a real-life mechanical device and vice-versa) is used a lot in their workflow, but a later ILM speaker informed us that even mentioning Motion Control when planning a VFX shot will anger all but the most patient director due to the set-up time for each take and re-take.

Finally, as for tools:

• They playblast all their work due to requiring much quicker turnaround on long scenes. I did see Depth of Field on a couple of shots, but Dan informed us this was a decision made after lengthy render times on Star Wars Episode II. The playblast was customized to auto-turn-off elements like skeletons and other helper objects.
• They created a time-of-day tool that adjusted the sun’s position via time sliders and longitude/latitude inputs.
• Their camera tool would display lens info and camera data such as speed, tilt and subject distance – all relevant for accurate real-world recreation. The tool also allowed easy switching between cameras and the ability to easily add a camera with all their custom presets.
• In addition, they also created a handheld cam-shake tool to simulate camera shake based on the movement of the cameraman.
• Extra frames are added either side of the shot as in real life to facilitate editing.
• Of most interest to me was the actual custom camera object itself, which consisted of a plane recreating the black bars above and below a wideframe shot. I’ve found that simply using the film-gate setting in Maya also shows what’s around the camera which can be quite distracting, though previous experiments with a black-bar model fail whenever the field of view changes. Additionally, the black bars also afforded a fantastic space in which to display info such as filename and frame-count etc.