Method and apparatus for determining the distance between an image and an object

• US 5,577,130 A
• Filed: 08/05/1991
• Issued: 11/19/1996
• Est. Priority Date: 08/05/1991
• Status: Expired due to Fees

• Current Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Original Assignee: Philips Electronics North America Corporation (Koninklijke Philips N.V.)
• Inventors: Wu, Hsiang-Lung
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Johns, Andrew W.
• Application Number:
  US07/740,279
• Time in Patent Office: 1,933 Days
• Field of Search: 382/1, 382/45, 382/48, 382/106, 382/153, 382/154, 382/291, 358/88, 358/105, 358/107, 356/12, 359/462, 364/561, 348/135, 348/139, 348/142
• US Class Current: 382/106
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A video camera is displaced to successive positions a displacement distance that differs from each preceding position by a factor of two wherein the camera image plane is at a constant distance from an object whose distance from the camera is to be estimated. The further that the camera is moved the more accurate the estimated distance. The camera is displaced a relatively small increment initially to provide an accurate estimate of the corresponding pixel of interest in the next camera position. The successive displacements increase rapidly to relatively large values, to which the estimated pixel disparity corresponds to give a relatively accurate camera to object distance estimate. The window size and position of each successive camera position for each pixel of the image can be accurately estimated to reduce potential error in pixel disparity estimation.

Claims

1. An apparatus for estimating the distance of an object from a video image comprising an array of pixels, said apparatus comprising:

imaging means having an image plane for acquiring in said image plane at a plurality of successive imaging positions, greater than two, a respective plurality of video images of a region of the object having a distance component in a given direction from said image plane, which distance component is to be determined, each image of said plurality of images having at least one pixel corresponding to a point on said object within said region;

means for automatically, without any action by an operator, predicting the location of a pixel window of a predetermined size in which said at least one pixel will be located in a next succeeding one of said plurality of video images;

means for estimating the value of the disparity of said at least one pixel in each pixel window, said disparity corresponding to the difference in position of said at least one pixel in a current image as compared to the position of said at least one pixel in an immediately preceding one of said plurality of images; and

means for determining the value of said distance component from the value of a last estimated disparity.

2. The apparatus of claim 1 wherein said imaging means includes means for acquiring said video images in a manner that said successive imaging positions are spaced from immediately preceding imaging positions by respective relative distances, each relative distance being equal to an immediately preceding relative distance multiplied by a given factor.

3. The apparatus of claim 2 wherein said given factor is an integer.

4. The apparatus of claim 2 wherein said factor is greater than one.

5. The apparatus of claim 1 wherein said imaging means comprises a digital video camera and means for displacing said camera in a plane parallel to said image plane.

6. The apparatus of claim 1 wherein each said successive position is spaced from the next preceding position by a spacing value, said imaging means including means for causing each said spacing value to increase in magnitude by a factor of two from the next preceding spacing value.

7. The apparatus of claim 1 wherein said means for prediction includes means to predict the position of a pixel in a successive image corresponding to a pixel in a preceding image to be positioned in a pixel window according to the relation nd(t)-1≦

d(t+1)≦

nd(t)+1 where d(t) is the estimated disparity of the corresponding pixel at iteration t of the preceding image, n is a factor greater than one and nd(t)±

1 represents the limits of the prediction window in which a pixel at iteration t+1 occurs having an actual disparity value d.

8. The apparatus of claim 7 wherein d(t+1)=2d(t) where the camera displacement is doubled in each iteration.

9. The apparatus of claim 1 wherein the disparity of a pixel of each successive image corresponding to a given pixel in the preceding image is given by the relation d(t)-1/2≦

d(t)≦

d(t)+1/2 where a(t) is the estimated disparity of a pixel at iteration t of an image and d(t) is the actual disparity of a pixel at iteration t of that image.

10. The apparatus of claim 9 wherein said means for prediction includes means to predict the position of a pixel in a successive image corresponding to a pixel in a preceding image to be positioned in a pixel window according to the relation nd(t)-1≦

d(t+1)≦

nd(t)+1 where d is the estimated disparity of the corresponding pixel at position t of the preceding image, n is a factor greater than one and nd(t)±

-1 represents the limits of the prediction window in which a pixel of an image taken at the position of iteration t+1 has an actual disparity value d.

11. An apparatus for estimating the distance of an object from a video image comprising an array of pixels, said apparatus comprising:

first imaging means having an image plane for creating at a first imaging means position a first video image of a region of the object, said first imaging means position having a distance component in a given direction from said image plane, which distance component is to be determined;

second imaging means having a corresponding image plane for creating at a second imaging means position at a first distance value from the first position a second video image of said region, said second imaging means position having said distance component in said given direction from said corresponding image plane;

means for automatically, without any action by an operator, predicting the location of a pixel window of a determined size at said second position in which at least one pixel of the first image is predicted to be positioned, said at least one pixel corresponding to a point on said object within said region;

means for estimating the value of a disparity of said at least one pixel in said window, said disparity corresponding to the difference in position of said at least one pixel in said second image as compared to the position of said at least one pixel in said first video image;

means for causing said second imaging means to create successive images at successive positions, which successive positions are at respective first distance values, for causing said means for predicting and estimating to successively respectively predict the pixel window location of each successive position and estimate the disparity value of said at least one pixel for each successive position; and

means for determining the value of said distance component from the value of the last estimated disparity.

12. A system for manipulating an object comprising:

a robot;

computer means for operating the robot;

imaging means coupled to the computer means for imaging said object, said imaging means having an image plane for creating at a plurality of imaging means positions, greater than two, a corresponding plurality of video images of a region of the object, said images each comprising a matrix of pixels, said region having a distance component to be determined, which distance component is in a given direction from said image plane; and

distance determining means coupled to the computer means and imaging means for estimating the distance component and therefrom, the distance between said robot and object, said distance determining means including;

means for placing imaging means at each said position for creating images of said object at each position such that all said image planes at said positions are coplanar, means for automatically, without any action by an operator, predicting a pixel window in each successively created image in which a given pixel of a preceding image is expected to occur and means for estimating a disparity of the given pixel in each said image, wherein the position of the imaging means at each successive position differs from the preceding position by a factor greater than one.

13. A method of estimating the distance of an object from a video image comprising an array of pixels, said method comprising:

acquiring in an image plane at a plurality, greater than two, of successive imaging positions a respective plurality of video images of a region of the object having a distance component in a given direction from said image plane, each image of said acquired plurality of images having at least one pixel corresponding to a point on the object within said region;

automatically, without any action by an operator, predicting the location of a pixel window of a predetermined magnitude in the image acquired at each of said successive positions in which said at least one pixel is located;

estimating the value of a disparity of said at least one pixel in each pixel window, said disparity corresponding to the difference in position of said at least one pixel in each successive image of said plurality of images as compared to the position of said at least one pixel in the immediately preceding image; and

determining the value of said distance component from the value of the last estimated disparity.

14. The method of claim 13 wherein said acquiring includes acquiring said video images in a manner that said successive positions are spaced from the immediately preceding positions by respective relative distances, each relative distance being equal to an immediately preceding relative distance multiplied by a given factor.

15. The method of claim 14 wherein said factor is two.

16. An apparatus for grasping and manipulating an object using a video image comprising an array of pixels to estimate the distance of the object, said apparatus comprising:

imaging means having an image plane for acquiring in said image plane at a plurality of successive imaging positions, greater than two, a respective plurality of video images of a region of the object having a distance component in a given direction from said image plane, which distance component is to be determined, each image of said plurality of images comprising an array of pixels including at least one pixel corresponding to a point on said object within said region;

means for automatically, without any action by an operator, predicting the location of a pixel window of a predetermined size in which said at least one pixel will be located in a next succeeding one of said plurality of video images;

means for estimating the value of the disparity of said at least one pixel in each pixel window, said disparity corresponding to the difference in position of said at least one pixel in a current image as compared to the position of said at least one pixel in an immediately preceding one of said plurality of images;

means for determining the value of said distance component from the value of a last estimated disparity; and

robot means constructed to grasp and manipulate said object; and

control means responsive to said determined value of said distance component for causing said robot means to grasp said object.

17. The apparatus of claim 16, wherein said imaging means includes a camera secured to said robot means for creating said images, said robot means including means for displacing said camera to said successive imaging positions.