In our laboratory set-up, an optical flow field is generated in real time from a sequence of gray-scale images. Velocities are computed by matching ``tiles'' of pixels in sequential frames, and not from differential relations of light intensity. This way, displacements much greater that one pixel can be found as easily as subpixel displacement. A coarse (40x30) optical flow field is computed for a sequence of video images at a real-time rate of 3 frames/second on a Silicon Graphics Indy workstation. From these estimates a discrete range map is computed from the discrete optical flow using the projection equations and rigid-body constraints. Together, the range and flow are used to produce a pose estimate, and the pose and range are finally used to fit a superellipsoid model to the data. With a 3D physical model, the translational and rotational dynamics of the object can be predicted within a Kalman filter. The optical flow for the next iteration is refined by feeding the expected projection of the object's surface back into the image plane. This strategy thus uses a combination of bottom-up measurements and top-down feedback. The laboratory implementation currently displays a 3D frame superimposed on a live video image showing the target object's pose and position. A color-coded image of the object will show the surface uncertainty on the object, allowing the user to manipulate the object intelligently, e.g. moving the object so as to bring less certain regions of the surface into view. (Figure 5.2).
S. Benoit, F.P. Ferrie