Method and apparatus for enhancing the dynamic range of stereo vision system

• US 20060140510A1
• Filed: 12/27/2004
• Published: 06/29/2006
• Est. Priority Date: 12/27/2004
• Status: Active Grant

• Current Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Original Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Inventors: Luo, Yun, Wallace, Jon K.
• Granted Patent: US 7,561,731 B2
• Time in Patent Office: Days
• US Class Current: 382/294
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 5/009 Global, i.e. based on properties of the image as a whole applied in cameras using an electronic image sensor H04N23/80, H04N23/70
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Systems and methods are provided for producing a stereo disparity map representing an imaged subject. A plurality of sensors (102 and 104) are operative to produce a plurality of images. Each image has an associated exposure according to an exposure level of an associated sensor. A system control (106) determines exposure levels for the plurality of sensors according to an exposure selection algorithm. A disparity processing component (112) is operative to generate a stereo disparity map from a plurality of images. An image merger (110) is operative to merge a plurality of images to produce a composite image having an increased dynamic range.

Claims

1. A system for producing a stereo disparity map representing an imaged subject, the system comprising:

a plurality of sensors operative to produce a plurality of images, each image having an associated exposure according to an exposure level of an associated sensor;

a system control that determines exposure levels for the plurality of sensors according to an exposure selection algorithm;

a disparity processing component that is operative to generate a stereo disparity map from a plurality of images; and

an image merger that is operative to merge a plurality of images to produce a composite image having an increased dynamic range.

2. The system of claim 1 wherein the image merger is operative to merge a plurality of stereo disparity maps to produce a composite stereo disparity map having an increased dynamic range.

3. The system of claim 1 wherein the image merger is operative to merge a first set of images, associated with a first sensor from the plurality of sensors, to create a first composite image and is operative to merge a second set of images, associated with a second sensor from the plurality of sensors, to create a second composite image, the disparity processing component being operative to combine the first composite image and the second composite image to provide a stereo disparity map.

4. The system of claim 1 wherein the image merger combines a plurality of images by summing the intensity values of corresponding pixels across the plurality of images.

5. The system of claim 4, wherein the image merger sums corresponding pixels only within a region of interest within the plurality of images.

6. An actuatable vehicle occupant restraint system comprising the system of claim 1.

7. The system of claim 1, the system further comprising an image analysis component that evaluates the plurality of images generated by the sensors and provides statistics relating to at least one of the plurality of images to the system control.

8. A method for producing a stereo disparity map having an enhanced dynamic range, comprising:

determining a first exposure level for a first image set according to an exposure selection algorithm;

generating the first image set at the determined first exposure level;

generating a first stereo disparity map from the first image set, the first stereo disparity map having an associated dynamic range;

determining a second exposure level for a second image set according to the exposure selection algorithm;

generating the second image set at the determined second exposure level;

generating a second stereo disparity map from the second image set, the second stereo disparity map having an associated dynamic range; and

merging the first stereo disparity map and the second stereo disparity map as to produce a composite stereo disparity map having an effective dynamic range greater than either of the associated dynamic ranges of the first disparity map and the second disparity map.

9. The method of claim 8, wherein the step of determining the second exposure level includes determining an area within the first stereo disparity map comprised by saturated regions and subtracting a predetermined value from the first exposure level if the area exceeds a threshold value.

10. The method of claim 8, wherein the step of determining the second exposure level includes determining an area within the first stereo disparity map comprised by dark regions and adding a predetermined value from the first exposure level if the area exceeds a threshold value.

11. The method of claim 8, wherein the step of determining a first exposure level comprises:

evaluating a first image from a previous iteration to determine the number of saturated pixels within the first image from the previous iteration;

determining a first difference between the determined number of saturated pixels and a target number of pixels; and

calculating the first exposure level as a function of an exposure level associated with the first image from the previous iteration and the determined first difference.

12. The method of claim 11, wherein calculating the first exposure level includes multiplying the determined difference by a predetermined constant to obtain a product and adding the product to the exposure level associated with the first image.

13. The method of claim 11, wherein the step of determining a second exposure level comprises:

evaluating a second image from a previous iteration to determine the number of saturated pixels within the second image from the previous iteration;

determining a second difference between the determined number of saturated pixels within the second image and a target number of pixels; and

calculating the second exposure level as a function of an exposure level associated with the second image from the previous iteration and the determined second difference.

14. The method of claim 13, further comprising:

determining if a first coverage range associated with the first exposure level overlaps a second coverage range associated with the second exposure level; and

determining a third exposure level for a third image set according to an exposure selection algorithm if the first and second coverage ranges do not overlap.

15. A method for generating a stereo disparity map having an enhanced dynamic range, comprising:

determining a desired exposure level for each of a plurality of images, the plurality of images including a first set of images and a second set of images;

generating each of the plurality of images at the desired level of exposure;

merging the first set of images to produce a first composite image;

merging the second set of images to produce a second composite image; and

generating a stereo disparity map from the first composite image and the second composite image.

16. The method of claim 15, further comprising evaluating at least one range value associated with the stereo disparity map and discarding at least a portion of the stereo disparity map if one of the at least one range value falls outside of an acceptable range.

17. The method of claim 15, wherein the step of determining a desired level of exposure includes determining an area within a first image from a set of images comprised by dark regions and raising the exposure level for a second image if the area exceeds a threshold value.

18. The method of claim 15, wherein the step of determining a desired level of exposure includes determining an area within a first image from a set of images comprised by saturated regions and lowering the exposure level for a second image if the area exceeds a threshold value.

19. The method of claim 15, wherein the step of determining a desired level of exposure for the plurality of images includes calculating an exposure level for a second image as a function of the difference between an average saturation level over an evaluated region of a first image and a desired average saturation level for the region.

20. The method of claim 15, wherein the step of determining a desired level of exposure for the plurality of images comprises:

determining the number of pixels falling within each of a plurality of intensity ranges in a region of interest within a first image;

comparing the determined number of pixels for each range to a series of desired values for the range; and

selecting an exposure value for a second image as to minimize the sum of the squared differences between the target values and expected values associated with at least one available exposure level.