See all NewsEngineering News

Coding Costumes

A new generative methods class bridges programming and theatre through virtual costumes

virtual costume

At first glance, the Chinese opera mask with red pom poms and pearls decorating an intricate crown looked just like an animation.

But it was not just animation. It was a virtual mask, following Northwestern student Yangdongling Liu's face and hand movements and controlled by code.

Liu, a PhD candidate studying physical chemistry, coded the costume in COMP_SCI 396: Generative Methods, a computer science class taught by Kate Compton that explores the intersection of programming and art. In class, students collaborated with theatre design students in the School of Communication to learn how theatre designers think about costuming and find parallels in their respective disciplines. Applying the artistic lessons they learned, students created virtual costumes that overlay graphic masks and hands on video and track body and face movements.

Blending computer science and art comes naturally to Compton, assistant professor of instruction in computer science at the McCormick School of Engineering and a generative artist. Last fall, Compton met with costume designer and Associate Professor of Costume Design Ana Kuzmanic to discuss a possible collaboration between theatre design students in the theatre department and students in Compton’s Generative Methods class, which started in winter quarter.

Professor Kate Compton programs a mask that tracks her face movements, with auto-wiggling eyes and a MIDI-controllable eyebrow slider.

“The collaboration was a way for Ana and me to communicate through our students and start to develop a common language that we could both understand, as well as doing it in an easy-going and non-intensive way during quarantine,” Compton said.

Rohith Jayaraman's mask was inspired by J.K. Rowling's "Harry Potter and the Goblet of Fire."

During a class demonstration on Zoom, Rohith Jayaraman’s mask was on fire. That is, it seemed to be. The virtual mask was inspired by a scene in J.K. Rowling’s Harry Potter and the Goblet of Fire when Sirius Black’s face appeared to Harry Potter in a fireplace.

As Jayaraman pulled up his fiery mask on video, spikey red flames radiated from his face. The code, he explained, reflected the emotions of the character, an artistic process he learned from theatre design students including Hayley Wallenfeldt, undergraduate designer and animator, and Benjamin Kress, an MFA costume designer, who fashioned individual elaborate visual presentations to illustrate main principles of theatre design. As Jayaraman talked, he controlled how quickly his mask’s flames flickered — fast when angry, and slow when happy.

“Art doesn’t come to me,” said Jayaraman, a first-year master of science in computer science student. “I’ve done a lot of programming, but I’ve never been able to bring something cool up on screen like this. There’s something different about creating a program versus creating something fun people can play with.” 

The eyebrows on one of Compton's coded masks are controlled by a MIDI slider.With code, costumes could even take on their own, asynchronous personalities. In some ways, virtual costumes become characters themselves.

“Imagine if your costume is moving, and then it winks at you,” Compton said. “There’s a character in your costume that’s very alive and sharing your face with you. It tears apart the connections we’ve assumed about the body.”

The possibility of virtual mask technology becoming more involved in theater productions is exciting, Compton said. “It’d be really amazing for a ‘Jeckyl and Hyde’ production to include a virtual mask projected onto an actor, and then someone offstage with a joystick taking control of the actor’s face.

“I think the real fun of these masks is that they are interactable live,” Compton added. “There are plenty of prerecorded videos used on stage, but this would let live actors and mask ‘puppeteers’ improvise playfully, like the Muppets do.”

Kate Compton
Another of Compton's masks, inspired by Voronoi diagrams.

This type of interdisciplinary collaboration is a hallmark of Northwestern’s mission. To Kuzmanic, there is much to learn from the ways people from different fields think creatively and approach problem-solving.

“On the surface, our fields might be perceived as opposites,” Kuzmanic said. “Fundamentally, while we speak different languages in terms of our respective disciplines, there are great similarities in our individual ways of generating and delivering information, and those similarities provide the foundation for meaningful collaboration.”