The Building Blocks of Success

Four students used the lessons learned in Embedded Systems in Robotics to construct an assistant that helps humans play Jenga.

Jenga is a game of physical and mental skill, but the objective of it is simple: remove one block from a stack of blocks and put it on top without making the stack fall.   

Anyone who has ever played Jenga knows it can get stressful as the game goes on. Four students in Northwestern Engineering's Master of Science in Robotics (MSR) program introduced a new level of complexity to the experience by creating a robotic assistant to help play the game. 

“Jenga is an interesting problem to tackle because precision in pulling and placing pieces is so crucial,” said Katie Hughes, one of four students to work on the project. “As it turns out, playing a full game of Jenga autonomously is an incredibly hard task for a robot.”  

Hughes worked with classmates Hang Yin, Liz Metzger, and Alyssa Chen on the project in Embedded Systems in Robotics, a project-based course that introduces students to a variety of software tools and concepts useful for robotics engineers. 

The students used a Franka Emika Panda arm to assist Jenga players by removing a block identified by a human and placing it on top of the stack without toppling the growing tower. 

“We were able to surpass our core goal by pulling and stacking 10 consecutive Jenga pieces by the end of the project,” Hughes said. “It was super exciting to accomplish more than we anticipated.”  

To achieve that, students integrated computer vision with motion control. The project served as an introduction to the Robot Operating System 2 (ROS 2), a key framework used by many robotics professionals and one strongly emphasized throughout the MSR program.   

“Learning ROS 2 through this class has been particularly valuable,” said Yin, who focused on integrating machine learning into the Jenga robot. “It has equipped me with a skill that will be useful in many robotics systems, both in industry and academia.”  

The students divided the project's tasks in half, with two of them tackling the robot’s movement and two handling its computer vision. The group said having teammates with different skill sets and specialities proved invaluable with the compressed timeframe to build a software stack to control the robot.  

“This project simply wouldn't have been possible for one person to complete in the time span we were given,” Hughes said. “I am so proud of how much we were able to accomplish together.”  

Yin said he's already applied lessons learned from the class in subsequent MSR courses. Hughes agreed, adding the opportunity to play Jenga with a robot helped her grow as a roboticist.   

“This was the largest and most complex project I had ever worked on,” Hughes said. “Now, I can look back and feel very proud of how much I accomplished in a relatively short amount of time.” 


McCormick News Article