Research Profile: Jack Tumblin removes motion blur in digital photography

December 11, 2006

It’s a problem that plagues parents at a soccer game, enthusiasts at an auto race, and even anyone with a slightly unsteady hand. Motion blur, the annoying streaks of movement across a still image, is a problem that has confounded generations of researchers. Now, research from Jack Tumblin, assistant professor of electrical engineering and computer science, and collaborators at Mitsubishi Electric Research Labs (MERL) in Boston, MA, aims to solve the problem to advance digital photography.

Most digital cameras are based on their film counterparts, opening the shutter once for each photo. Any movement in the scene while the shutter is open becomes blurry during the exposure, a common problem for all levels of photographers. “Essentially everything is moving to some extent all the time,” Tumblin explains. “We don’t hold the camera perfectly still and we don’t have subjects that are still.”

While there are several methods to reduce or eliminate motion blur by deconvolution, the best are very complex yet still provide noisy results. The process is flawed because some components of the photo inside the blur are completely lost. “It’s essentially mathematical guesswork,” Tumblin says. “It’s an informed guess, but you can’t reliably determine what the deblurred image actually is. As you’re missing some information, your best guesses may still be wrong.”

Rather than improve deconvolution, Tumblin and colleagues Ramesh Raskar, senior researcher at MERL, and Amit Agrawal, visiting researcher at MERL, made fundamental changes to the way a photo is taken. Using a specialized electronic shutter on an ordinary digital camera, they were able to flutter the camera shutter open and closed multiple times as they took a single photograph. Manufacturers may later incorporate this rapid ‘flutter’ into the electronic image sensor itself to eliminate this external shutter.

A pseudo-random pattern for opening and closing the shutter essentially chops up the motion blur. That process allows the digital image file to retain all of the high-frequency spatial information that would have otherwise been lost in ordinary motion blur.

Instead of complex deconvolution methods, the researchers set up a matrix with the image and the coded flutter pattern, and then invert the matrix. The result is a photo that captures sharp details on moving objects, such as the license plate on a speeding cab. (For example pictures and technical details, see http://www.merl.com/people/raskar/deblur/.)

This result is part of Tumblin’s ongoing work in computational photography, a new field that aims to rethink how visual experiences are captured. With novel digital devices and new capture methods, we can compute a better result than we can directly capture on film. To support this work, Dave Story, VP of Adobe Systems, Inc. and lead engineers in the PhotoShop group recently decided to make a new unrestricted gift of $25K to support Tumblin’s efforts. Researchers in this new field envision a fundamental change in how we capture images – one that moves away from devices that mimic film functionality toward devices that work more like the human eye, capturing relevant, meaningful moments in time.