Helping Wheelchairs Navigate the World Around Them

Mahdieh Nejati Javaremi (MSR '15) was looking for a challenge when, midway through her time in Northwestern's Master of Science in Robotics (MSR) program, she contacted Brenna Argall.

Argall was one of Javaremi's first professors in the program, and her research was focused on the intersection of artificial intelligence, rehabilitation robotics, and machine learning. That was the intersection where Javaremi decided she wanted to be, and so she asked to do her final MSR project with Argall.

That project was a smart wheelchair that was retrofitted with a computer and different sensors to help it sense the world around it and act intelligently. That wheelchair project wound up being more than a class project. It set a professional path for Javaremi that extended beyond MSR.

Thanks to that project, Javaremi is still working with Argall, this time for her PhD as a graduate research assistant at the Shirley Ryan Ability Lab where she once again is using her knowledge to enhance rehabilitation robotics.

Abhishek Patil (MSR '16) saw the work Javaremi did on the smart wheelchair and felt he could enhance it even further. So for his final project, he too worked with the smart wheelchair in an effort to help wheelchair-bound people navigate the world around them. Today, he is a research engineer at the Honda Research Institute, where he develops autonomous driving technologies.

Patil and Javaremi recently took time to look back on the smart wheelchair project and discuss what they learned from the experience and how it influenced what they are doing professionally today.

Why were you interested in doing a smart wheelchair project?

MJ: One of the reasons I was interested in robotics was because of assisted robots, that's where my main passion was. The smart wheelchair that we had was a regular powered wheelchair that was retrofitted with a computer and different sensors. The main goal was to provide assistance in a shared, controlled way. To do that, the robot needs to be able to understand its surroundings.

When I joined the lab, there was already a doorway detection algorithm and docking algorithm. What we didn't have in the lab was a ramp detection algorithm for people going up and down ramps or changing speeds while on a ramp.

Today, our lab essentially has a wheelchair track. We have a doorway, different ramps, and dropoffs. We have these different props that allow us to have controlled experiments. The fact that we have a ramp in the lab grew from that initial ramp detection project.

AP: This project was the best way for me to pursue my interest in autonomous navigation technologies, including perception and motion-planning components. I was happy to see that my contribution was going to positively impact the lives of other people, particularly people with disabilities.

What did you learn from your work on the wheelchair?

MJ: I honed a lot of the skills that I learned during the program. I actually hadn't taken any computer vision courses, so I had to learn that on my own as I went.

One thing that was interesting was that I started working in simulation and then moved on to a real robot. Transferring the code that worked well in the simulation to then have it work on an actual robot was a challenge. In the simulation, you have a simplified model, so I learned how you deal with that.

AP: This was a real-world project where dynamic properties of the wheelchair model had to be thought of and constructed from scratch. I had to test a few dynamic models in simulation to determine the optimal one. Also, the code base for this project was in C++, and my proficiency was in Python, so it was challenging to understand the existing code.

I learned how to model a dynamic object in ROS Gazebo just from a picture of the actual wheelchair. To do that, I created a 3D model using Solidworks and then exported it to Gazebo with simulating dynamic properties of various parts of the wheelchair.

I also got a glimpse into how innovative research happens, from identifying and understanding challenges to digging deep into implementation and setting feasible milestones. These are all important aspects of innovative research. Additionally, planning and communication are vital to ensure a project is completed efficiently.

How would you describe your overall MSR experience?

MJ: It was really great. In addition to my classes, I learned a lot from my cohort. MSR was an accelerated learning experience, and it was challenging, but it was totally worth it.

Northwestern has faculty who are renowned in our field, and the MSR program gives you access to those faculty as well as different robots. A lot of the program is structured, but it is designed to give you the freedom to carve out your own specialization. It was a great experience, and it gave me a lot of skills that made my PhD a little easier.

AP: The MSR program helped me develop my passion for robotics by teaching me skills I needed to be ready to work in the robotics industry, from programming in Python and ROS to portfolio development, time management, and communication skills.