Visual Tracking
This happens to be a large part of my research. Visual sensing in general is one my principle focus areas, (look at the HRI page for more on that) but my Master's work was on visual servoing, which is a technique of controlling robots or robot manipulators (e.g. arms, legs) using visual feedback. Particularly, I focused on getting Aqua to follow a target underwater. The same principle would work on terrestrial environments, but it has not been implemented on the robot yet.
While most of the tracking methods implemented on the robot are existing ones, we are currently working on developing new algorithms suitable for the underwater domain. Existing algorithms include a naive color segment (or blob) tracker, brute-force histogram-matching tracker and two variants of the mean-shift tracker (with and without background weighting). Stereo tracking (for distance/depth estimation to control robot speed) and multi-target trackers are currently in the works.
Almost all of the vision work has been implemented in C++. The VXL library has been used as the base for all of that. VXL provides a suit of libraries for computer vision and related research in C++. Matlab, as always, is used to prototype the algorithms. Although, that does not happen too often in my work.
More information on the tracking algorithms, their integration with the visual servoing system and a comparative study of their performances in underwater environments can be found in the publications page.
Motion tracking
The idea for this came up quite unexpectedly, when looking at video footage of the robot following scuba divers. We noticed that the flippers of a diver was oscillating in a quite uniform way, with a constant periodicity. This gave us the idea that somehow, we could look at the exact location of a diver's location in successive image frames and extract the periodic intensity signature it registers. Since the flippers do not oscillate at high frequencies (usually between 1 and 2 Hz), doing a Fourier transform on such an intensity signal would give us an amplitude spectra with large responses in the low-frequency components. And it did precisely that.
| Aqua follows a diver (left) and the FFT plot of the video-slice containing the diver. |
In effect, we have a cylindrical shape through a sequence of images going backward in time, and we extract just one intensity value for each slices that makes up this 3D cylinder. This gives us an array of intensity values which we call ann intensity vector. Many such vectors are formed in a sequence of frames, and the ones that exhibit high-responses in the low-frequencies, are taken as a potential location of a diver. We combined this with the visual tracker described in the preceding section, and have come up with a very strong algorithm for tracking biological motion in general. This was published in IROS 2007 at San Diego, and the paper can be found in the proceedings page or in the MRL publications page, like always.