November 20, 2017

Bringing the AZ Capitol to Life with DV Magic

A few weeks into the Fall semester, Dr. Bolman emailed me and asked “What do you think about making a talking CGI Arizona Capitol building?” to which I said “Huh?” Turned out Thrillist had named our capitol building one of the ugliest in the nation, and Phoenix’s own “Hip Historian” Marshall Shore had taken exception to this. He had mentioned to Dr. Bolman the idea of doing a video rebuttal featuring the puppet version of the Hip Historian interviewing the maligned capitol about the Thrillist article. At which point Dr. Bolman brought it to me and my team of students to make it happen.

After a couple of weeks of wrangling my schedule, Marshall’s, and that of puppeteer Stacey Gordon of Sesame Street fame, we finally got everyone lined up to produce the necessary elements. Marshall and photographer Marilyn Szabo got us an excellent shot of the capitol to use as the basis for our animation. Marshall also recruited veteran Phoenix actor Greg Lutz to voice the embattled building. (Greg himself is no stranger to UATDV productions, appearing in Jacob Glass’ “Afterlife” and recently filming a stint as the mayor of the titular town in “Help Falls.”)

Now by week 6 of the semester, students are pretty nose-down into their own projects – and our production slate this semester is rather heavy, shooting 2 new films while completing post-production on 2 others – so I took it upon myself to do the lion’s share of the work for this. I met Greg and Marshall early one Friday morning in our DV Studio and took full advantage of our new “Recording” light we’re sporting outside the door. We did a few takes, making adjustments and having some good laughs. Greg and Marshall were consummate professionals, and having them in studio together (rather than recording them separately and editing their lines together) gave the dialogue an authentically conversational rhythm.

A couple of hours later Stacey Gordon joined us in the Greenscreen Studio where students Jake Turocy, Paul Lopez, and Gabe Vigil had set up our Blackmagic URSA Mini camera and lit the puppet Marshall to match the photo of the capitol.

Student Jake Turocy films puppeteer Stacey Gordon

Turns out filming a puppet is very technical, so we all learned quite a bit from Ms. Gordon’s experience and expertise. That alone made the project worth it! Any time we can expose to students to professionals who can add to their education, it’s a win.

Once all our elements had been recorded, then I was ready to bring it all together. Tools used for the final product:

  • Adobe Audition – used for sound editing and sweetening, including adding reverb to Greg’s lines as the capitol
  • Nuke – our visual effects workhorse was used to pull the key on the puppet shots (“pulling a key” means to remove the greenscreen and create transparency)
  • Adobe After Effects – used to animate the capitol and bring all the elements together

Now I could have completed the visuals for the project entirely in After Effects if I wanted to, and in fact the first version I sent to Marshall for approval later in the day after the shoot was executed in AE. But I opted to use Nuke for the keying of the greenscreen footage due to how much better the quality of the keyers are in Nuke versus those in AE. In particular, puppet Marshall’s white hair and goatee were problematic – hair is always difficult to key, and lighter hair is perhaps the worst. Nuke’s Image-Based Keyer is my go-to for preserving hair detail and it worked great here. (You can check out my Nuke keying tutorial for more info, if you’re so inclined.)

To animate the capitol, I knew I needed to find the most expedient way to handle the lipsynch since there’s so much dialogue. To that end I used After Effects’ Keyframe Assistant and its built in tool “Convert Audio to Keyframes” to generate a series of keyframes based on the relative volume of Greg’s voice track. From there I started exploring options for how to use these keyframes to create some kind of mouth animation. I started digging into AE’s “Distort” category of effects and found “CC Split” which creates a kind of zipper animation with 2 anchor points and a single slider to control the width of the split. Positioning the anchor points to become the corners of the capitol’s “mouth” and using a simple expression to wire the amplitude keyframes to the split slider – I immediately saw the potential for a mouth animation that mimicked how the practical puppet’s mouth moved.

From there, I used After Effects’ MeshWarp effect to add a series of distortion handles on the capitol building and with them I was able to keyframe some simple expressions to make the capitol mad, sad, happy, etc. Layering this on top of the mouth animation got me the effect I was looking for. Finally, I added the eyes – which actually was the first thing I conceived but I left it until last after I had tackled everything else. To me, the two brick squares on either side of the building’s front section seemed like closed eyes. With some masking, I was able to create the illusion that those squares opened like eyelids, revealing puppet-like “googly eyes” underneath. Some simple animation was added to the pupils to move them appropriately, and blinks were added to give the building a little more life.

The finished video has since been featured on Downtown Phoenix Journal and AZ Central. People seem to be enjoying it, and it’s certainly getting people talking about one of Arizona’s architectural treasures.

Arizona State Capitol, Ugly? from RetroSpectacular on Vimeo.

Leave A Comment