Generating categorical depth maps using passive defocus sensing

• US 5,793,900 A
• Filed: 12/29/1995
• Issued: 08/11/1998
• Est. Priority Date: 12/29/1995
• Status: Expired due to Fees

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Stanford University
• Inventors: Nourbakhsh, Illah Reza, Andre, David
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US08/581,245
• Time in Patent Office: 956 Days
• Field of Search: 382/106, 382/107, 382/153, 382/154, 382/255, 382/262, 382/263, 356/2, 356/376, 348/139, 348/348, 348/355
• US Class Current: 382/263
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/236 using varifocal lenses or mirrors
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

A method is disclosed for generating a categorical depth map of a scene using passive defocus sensing. In a preferred embodiment three synchronized CCD cameras focused at different distances detect three images of the same scene. An image processor partitions the images into an array of regions and calculates a sharpness value for each region. The sharpness value for a region is calculated by summing over all pixels (x,y) in the region the absolute difference in the intensity value of a pixel (x,y)( with pixel (x-k,y-l), where k and l are constants. The image processor then constructs a depth map of the scene by determining for each region the image with the greatest sharpness in that region. An application of the invention to a mobile robot control system is described in detail. Among other applications, the method may be used for collision avoidance, object detection, and speed measurement.

Claims

1. A method for generating a categorical depth map of a scene, the depth map comprising m depth map regions, where m≧

1, the method comprising the steps of;

detecting n images of the scene using n cameras focused at different distances, where n>

1;

dividing each of the n images into m predetermined two-dimensional image regions such that each depth map region corresponds to n similar image regions taken from the n images;

determining a sharpness value for each region of the m image regions in each image of the n images;

identifying, for each depth map region, an image number of a sharpest image selected from the n similar image regions taken from the n images, wherein the sharpest image has a maximal sharpness value; and

assigning a categorical depth value to each depth map region of the m depth map regions, where the depth value for a depth map region is determined from an image number of the sharpest image identified for the depth map region.

2. The method of claim 1 further comprising the step of synchronizing the n cameras so that they simultaneously detect the n images of the scene.

3. The method of claim 1 further comprising the step of orienting the n cameras so that the n images represent very similar but slightly different perspectives of the scene.

4. The method of claim 1 further comprising the step of using at least one beamsplitter to ensure that the n cameras view the scene from the same perspective.

5. The method of claim 1 wherein the step of determining a sharpness value for each region of the m image regions comprises:

finding, for each pixel in the region, the absolute difference between the intensity value of the pixel and the intensity value of a nearby pixel; and

summing together the absolute differences found for all the pixels in the region.

6. The method of claim 5 wherein the nearby pixel is a pixel horizontally displaced by k pixels and vertically displaced by l pixels, where k and l are integers, one of which is not zero.

7. The method of claim 1 wherein the step of assigning a categorical depth value to each depth map region comprises finding among the n image regions corresponding the depth map region a region having a maximal sharpness value.

8. A device for producing a categorical depth map of a scene, the depth map comprising m depth map regions, where m≧

2, the device comprising;

n camera means for detecting n images of the scene, where the n camera means are focused at different distances, where n>

1;

image processing means for dividing each of the n images into m predetermined two-dimensional image regions such that each depth map region corresponds to n similar image regions taken from the n images, and for determining a sharpness value for each region of the m image regions in each image of the n images;

comparing means for assigning a categorical depth value to each depth map region of the m depth map regions, where the depth value for a depth map region is determined from an image number of a sharpest image selected from the n similar image regions taken from the n images.

9. The device of claim 8 further comprising a synch means for synchronizing the n camera means with each other so that they simultaneously detect the n images of the scene.

10. The device of claim 8 wherein the n camera means are oriented so that the n images represent very similar but slightly different perspectives of the scene.

11. The device of claim 8 further comprising a beam splitting means for ensuring that the n cameras view the scene from the same perspective.

12. The device of claim 8 further comprising a filtering means for reducing the intensity of light detected by the n camera means.

13. The device of claim 8 wherein the image processing means comprises a first means for finding, for each pixel in the region, the absolute difference between the intensity value of the pixel and the intensity value of a nearby pixel;

and a second means for summing together the absolute differences found for all the pixels in the region.

14. The device of claim 13 wherein the nearby pixel is a pixel horizontally displaced by k pixels and vertically displaced by l pixels, where k and l are integers, one of which is not zero.

15. The device of claim 8 wherein the comparing means comprises a means for finding, among the n image regions corresponding the depth map region, a region having a maximal sharpness value.

16. The device of claim 8 further comprising n intensity control means for controlling the amount of light detected by the n camera means.

17. The device of claim 8 used for detecting the presence of an object.

18. The device of claim 8 used for detecting the speed of an object.

19. The device of claim 8 used to assist a visually impaired person.

20. A mobile robot comprising:

n camera means for detecting n images, where the camera means are focused at different distances;

image processing means for dividing each of the n images into m predetermined two-dimensional image regions such that n similar image regions taken from the n images are in correspondence with each other, and for determining a sharpness value for each image region in each image, where the sharpness value is given by the formula ##EQU2## comparing means for generating a depth map by comparing the sharpness value of each region with the sharpness value of a corresponding similar region in another image; and

motion control means for controlling the movement of a robot in response to information contained in the depth map.