Optical flow is the velocity field induced on the retina by the relative motion between a viewer and an object in its visual field. As such it encodes important perceptual cues with respect to the motion and structure of objects in a scene. The advent of low-cost sensors coupled with high-performance computing power has re-kindled interest in both the determination of the optical flow field and its interpretation in terms of scene structure. The context of this research is the characterization of three-dimensional shape given prior knowledge in the form of a parametric model. In this scenario an operator presents a target object to a video camera and moves it according to the computer's suggestions for new viewpoints (using a strategy derived from the autonomous explorer). Our goal is to correctly recover the 3-D motion and structure of the object from the resulting flow and to minimize the ambiguity of this interpretation by using constraints derived from the structure of the model and feedback provided to the operator. In our laboratory set-up, an optical flow field is generated in real time from a sequence of gray-scale images. Velocities are computed by matching ``tiles'' of pixels in sequential frames, and not from differential relations of light intensity. This way, displacements much greater that one pixel can be found as easily as subpixel displacement. A coarse (40x30) optical flow field is computed for a sequence of video images at a real-time rate of 3 frames/second on a Silicon Graphics Indy workstation. From these estimates a discrete range map is comptued from the discrete optical flow using the projection equations and rigid-body constraints. Together, the range and flow are used to produce a pose estimate, and the pose and range are finally used to fit a supersllipsoid model to the data. With a 3D physical model, the translational and rotational dynamics of the object can be predicted within a Kalman filter. The optical flow for the next iteration is refined by feeding the expected projection of the object's surface back into the image plane. This strategy thus uses a combination of bottom-up measurements and top-down feedback. In the near future, the implementation will display a 3D frame superimposed on a live video image showing the target object's pose and position. A color-coded image of the object will show the surface uncertainty on the object, allowing the user to manipulate the object intelligently, e.g. moving the object so as to bring less certain regions of the surface into view.
S. Benoit, F.P. Ferrie
Laboratory set-up for building object models from optical flow measurements