Depth sensing camera systems and methods

• US 5,930,383 A
• Filed: 09/24/1996
• Issued: 07/27/1999
• Est. Priority Date: 09/24/1996
• Status: Expired due to Fees

• Current Assignee: Yishay Netzer
• Original Assignee: Yishay Netzer
• Inventors: Netzer, Yishay
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Dang, Duy M.
• Application Number:
  US08/718,829
• Time in Patent Office: 1,036 Days
• Field of Search: 382/154, 382/106, 382/286, 382/141, 382/148, 382/293, 382/201, 382/276, 382/285, 356/375, 356/1, 356/4, 356/2, 356/5, 356/141, 356/352, 356/98, 356/108, 356/125, 348/47, 348/217, 348/218, 348/42, 348/139, 348/43, 348/46, 348/48, 348/50, 348/207
• US Class Current: 382/154
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures

Abstract

This invention relates to a method and apparatus for sensing a three-dimensional (depth and illuminance) image of a scene. It is based on the inverse-square law relating the incident brightness on an area illuminated by a light point source, to its distance from the point source. In the preferred embodiment of the invention the scene is sequentially illuminated by more than one light point source each at a pre calibrated location in the reference coordinate system. The resulting reflections from the field of-view are sensed by a stationary digital camera that maps each scene element into a corresponding image pixel, to provide a 2-dimensional brightness map that contains the photometric values of each image pixel for each specific illumination. Each pixel photometric value depends on the illumination incident on the corresponding scene element which, in itself, is further determined by the element inherent Lambertian reflectance-coefficient at the illumination wavelength, the element orientation relative to the coordinate system, and the element illuminance as determined by the point source brightness and the distance separating the point source and the scene element. Each brightness map is different from its sequel due to the differing point-source locations. By manipulating the brightness maps the spatial location of each scene element relative to the fixed point sources is determined, thus yielding a depth-image as well as a brightness-image.

Claims

1. A method for deriving depth information associated with pixels of an image sensed by a digital camera, each pixel corresponding to a surface element of a subject scene, the method comprising the steps of:

(a) generating a first intensity map of the subject scene while substantially the entirety of the subject scene is illuminated by a first light source positioned in a first known geometrical relation to the digital camera;

(b) generating a second intensity map of the subject scene while substantially the entirety of the subject scene is illuminated by a second light source positioned in a second known geometrical relation to the digital camera;

(c) generating a third intensity map of the subject scene while substantially the entirety of the subject scene is illuminated by a third light source positioned in a third known geometrical relation to the digital camera;

(d) generating a fourth intensity map of the subject scene while substantially the entirety of the subject scene is illuminated by a fourth light source positioned in a fourth known geometrical relation to the digital camera; and

(e) processing the measured intensity values of corresponding pixels of said first, second, third and fourth intensity maps to derive information relating to the depth of the corresponding surface element of the subject scene, said processing being based upon an inverse square relation between distance and intensity.

2. The method of claim 1, further comprising employing a smoothing algorithm for rejecting local depth errors due to specular reflection.

3. The method of claim 1, further comprising generating additional intensity maps of the subject scene while the subject scene is illuminated by additional light sources each positioned in different known geometrical relation to the digital camera so as to minimize the chance of erroneous depth calculation due to specular reflection.

4. The method of claim 1, further comprising generating a calibration intensity map of the subject scene illuminated only by background illumination, and wherein said processing includes subtracting said calibration intensity map from each of said first, second, and third intensity maps.

5. The method of claim 1, wherein each of said first, second, third and fourth light sources approximates to a point source with respect to the area of the object scene.

6. The method of claim 1, wherein said processing derives information relating to the orientation of said corresponding surface element of the subject scene.

7. The method of claim 1, wherein said processing derives information relating to the Lambertian reflection coefficient of said corresponding surface element of the subject scene.

8. A depth sensing camera system for generating three-dimensional information about an object scene, the system comprising:

(a) a digital camera having a field of view including the object scene;

(b) a first light source positioned in a first known geometrical relation to said digital camera for illuminating substantially the entirety of the object scene;

(c) a second light source positioned in a second known geometrical relation to said digital camera for illuminating substantially the entirety of the object scene;

(d) a third light source positioned in a third known geometrical relation to said digital camera for illuminating substantially the entirety of the object scene;

(e) a fourth light source positioned in a fourth known geometrical relation to said digital camera for illuminating substantially the entirety of the object scene;

(f) a control system for sequentially activating each of said light sources such that said digital camera generates a first image of the object scene illuminated by said first light source, a second image of the object scene illuminated by said second light source, a third image of the object scene illuminated by said third light source, and a fourth image of the object scene illuminated by said fourth light source; and

(g) a processor configured to process said first, second, third and fourth images based upon an inverse square relation between distance and intensity to derive depth information relating to at least some points of the object scene.

9. The system as in claim 8, wherein said first, second and third light sources are arranged substantially collinearly.

10. The system as in claim 8, wherein each of said light sources approximates to a point source with respect to the area of the object scene.

11. The system of claim 8, wherein each of said light sources is an LED.

12. The system of claim 8, wherein each of said light sources is an illuminated optical fiber.

13. The system of claim 8, wherein said digital camera is a line detector employing a linear array of detector elements, the system further comprising means for generating relative movement of the object scene relative to said digital camera in a direction perpendicular to said linear array.