Bringing Predator-Prey Simulations to Life

When Senthil Palanisamy (MSR '20) considered his final project for Northwestern Engineering's Master of Science in Robotics (MSR) program, he was inspired by Professor Malcolm MacIver, whose research focuses on predator-prey relationships and helping robots sense the world around them. 

"Having published some great results on behaviors observed in simulation, Professor MacIver and his lab wanted to test if their results transfer over to real-world animals," Palanisamy said. He worked with MacIver to create high-speed image stitching and tracking that made it possible for the lab to replicate MacIver's simulations with a mouse in a maze.

Palanisamy reflected on the project, his time in MSR, and how the program prepared him to land a job where he utilizes computer vision to help reduce labor exploitation in third-world countries. 

What was the goal of your project?

My project was about providing computer-vision engineering support for a neuro-biological experiment. To accomplish this, a maze was set up for a mouse. The mouse was the prey and was tracked at high speeds using four cameras. This tracking information was fed to a robot, which acted as the predator and needed real-time planning information about the mouse’s location. My work primarily revolved around setting up the engineering infrastructure to support these predator-prey experiments. 

What made you interested in this project?

I have always loved cameras and developing computer-vision algorithms. Among all the sensors that we have at our disposal for robots, the camera is arguably the most powerful and most insightful.

How do you think the project turned out?

The project was a success. I showed mice tracking at more than 100 frames per second (FPS) using CUDA-enabled algorithms. I also implemented a deep-learning model to track the pose of the mice for more powerful offline analysis. Since speed is a vital aspect of the system, I did a lot of multi-threading and toward the end, the system usage was almost 700 percent. The engineering infrastructure I set up helps the lab verify its simulation results on actual animals, thereby establishing the validity of their simulation results.

Not only did I help get the infrastructure in place, but I set up my part in a user-friendly way. Anybody in the lab who has access to the hardware should be able to set up the stitching system for this experiment without having to know anything about the code or algorithmic details. Overall, I was able to set up a pretty sophisticated system from scratch. 

What were the biggest challenges that you faced along the way?

The image stitching problem had a few hidden demons that I never foresaw. It involved a lot of small improvements to ultimately get the perfect image stitching. At the start of the project, I thought a speed above 100 FPS was very unrealistic. But I had some lessons to learn about the power of CUDA algorithms. The lab also had a pretty sophisticated system to support such high-speed image capturing.  

It's quite amazing what you can accomplish if you keep your mind open to possibilities and absorb all the necessary knowledge around you. At the end of the project, I was able to look back and say that I did something I initially thought was not possible. That tells me this was a great project to enrich my skills.

How did this project incorporate lessons you learned throughout your time in MSR?

The C++ experience I gained from some of my courses helped me a lot, and ROS is a universal robotics requirement. A course on advanced computer vision helped me refine my 3D-vision knowledge, which was helpful when building the image stitching system. Not all the courses are about the immediate application. Some courses definitely refined my thought process to help me think better, which is hard to quantify. The program also helped me take my ability to work independently to a new level. 

What were the most important things you learned from this project?

Persistence pays off. Be on the constant lookout for how you can make things happen. You will discover new possibilities that you didn't think of previously.

What are you doing now after graduating from MSR?

I had four job offers and decided to join a company called SoftWear Automation as a computer vision engineer. The company is trying to automate the process of t-shirt manufacturing, and my role will be to support the automation process with computer vision. It's a pretty exciting problem and has the potential to end labor exploitation in the manufacturing process in third world countries.

What advice would you give a prospective student considering MSR?

Work hard. Learning compounds over time. Every hour you put in graduate school might distinguish an exceptional person from a good one. I am also a strong proponent of the idea that a degree is not the only indication of someone's capabilities — it is what you make out of your degree that matters. Don't shy away from picking difficult courses if you think they can benefit your long-term career. Don't try to be a jack of all trades. MSR's mandatory courses give you good exposure to cover the breadth of robotics, but choose the rest of your courses wisely so you can establish yourself as an expert in your area of interest.