Moving Mobility Forward

Marshall Johnson wants to take life-changing technology designed to help people without the use of their hands and make it even more transformative. 

Johnson used his final project in Northwestern Engineering's Master of Science in Robotics (MSR) program to advance what’s called sip-and-puff (SNP) technology to teach a machine to anticipate a user’s intentions. 

SNP currently is used to help increase people’s mobility in a wheelchair or their ability to use a computer mouse. It measures changes in air pressure on an integrated “straw” through the sucking of a sip or the exhalation of a puff and links it to specific tasks. For example, a strong sip could mean “stop,” while a soft puff could mean “turn left.”  

Johnson’s final project aimed to increase the speed at which a machine could learn from users’ various sips and puffs to create an advanced smart wheelchair.  

“The intimate connection between medicine and technology has always been fascinating to me,” Johnson said. “Having grown an affinity for machine learning (ML) throughout my time in the MSR program, I wanted to focus my final project on a challenging ML application that could positively impact someone’s life, particularly through medicine.” 

His project laid the foundation to do just that.  

In addition to ML, Johnson leveraged knowledge of robotic systems and navigation that he developed in MSR to create a program connected to a series of virtual mazes. Users provided their own SNP inputs while a computer did the actual driving. As users ventured through each maze, the machine learned to better predict what they were intending to do. 

Participants were then set loose on another randomly generated maze, this time in total control of the virtual wheelchair. Johnson's program accurately recognized users’ decisions to move, stop, and turn more than 90 percent of the time, including several flawless trials. 

“This project highlighted the necessity to always keep the user at the forefront throughout development,” Johnson said. “Solutions immediately apparent were not always the best solution because they did not have the user in mind. 

"One of the goals achieved through this project was allowing a user to create their own symbolic language, a language that reflects their personal preferences and capabilities. This custom interface mapping real-world movements to preferences and capabilities hopefully made it easier for a user to control the virtual wheelchair, and in turn, would make controlling a wheelchair in real life more intuitive as well." 

Johnson, who will graduate later this year, felt the MSR curriculum was structured in a way that let him build on skills he learned earlier in the program and bring them all together as part of his final project. The program taught him how to pursue certain paths — and also how to identify when a path is not leading him in the right direction.  

“My final project challenged me to both grow my existing skills and learn new ones,” he said. “It also pushed me to rapidly iterate on ideas and brainstorm others during regular meetings with my advisors. I was able to 'fail fast’ in order to identify what worked best and arrived at a robust proof of concept by the end of the summer.” 

Johnson knew he would not be able to perfect his new concept within the confines of the final project, but his goal was to lay the foundation for a future student to build upon. Thanks to his advisers Matthew Elwin and Ola Kalinowska, he was able to make that happen. 

“Toward the end, I started packaging it so someone else could ramp up quickly to build on the project in the future,” he said. “I am hoping someone picks up where I left off to add more features and increase its robustness.” 

The project itself, coupled with his experiences in the MSR program, helped shape Johnson’s post-graduation plans. His hope is to pursue a career that brings humans and robotics together to help address real-world healthcare challenges.  

“My capstone project helped me grow and equipped me with foundational skills I will leverage long into my career as a roboticist,” he said. “It further grew my passion to use my skills to serve others around the world, and I hope to continue to pursue this mission in the professional world soon.”