Method and apparatus for three-dimensional shape estimation using constrained disparity propagation

• US 7,561,732 B1
• Filed: 02/04/2005
• Issued: 07/14/2009
• Est. Priority Date: 02/04/2005
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Owechko, Yuri, Medasani, Swarup, Srinivasa, Narayan, Boscolo, Riccardo
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): PERUNGAVOOR, SATHYANARAYA V
• Application Number:
  US11/051,592
• Time in Patent Office: 1,621 Days
• Field of Search: 382/154
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A method, an apparatus, and a computer program product for three-dimensional shape estimation using constrained disparity propagation are presented. An act of receiving a stereoscopic pair of images of an area occupied by at least one object is performed. Next, pattern regions and non-pattern regions are detected in the images. An initial estimate of śpatial disparities between the pattern regions in the images is generated. The initial estimate is used to generate a subsequent estimate of the spatial disparities between the non-pattern regions. The subsequent estimate is used to generate further subsequent estimates of the spatial disparities using the disparity constraints until there is no change between the results of subsequent iterations, generating a final estimate of the spatial disparities. A disparity map of the area occupied by at least one object is generated from the final estimate of the three-dimensional shape.

Claims

1. A method for three-dimensional shape estimation using constrained disparity propagation, the method comprising an act of causing a processor to perform operations of extracting image features, wherein the operations of extracting image features comprises the acts of:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the act of using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

processing the final estimate to provide signals, wherein the signals comprise enable and disable signals of a vehicle;

generating a disparity map of the area occupied by at least one object from the final estimate of the three-dimensional shape; and

processing the disparity map with at least one classification algorithm to produce object class confidence data, said classification algorithm is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

2. The method of claim 1, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area.

3. The method of claim 1, wherein the method further comprises an act of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

4. The method of claim 2, wherein the signals comprise airbag enable and disable signals.

5. The method of claim 1, wherein an act of data fusion is performed on the object class confidence data to produce a detected object estimate.

6. An apparatus for object detection comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images, and an output coupled with the processor for outputting information based on an object estimation, wherein the computer system further comprises means for:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the means for using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

processing the final estimate to provide signals, wherein the signals comprise enable and disable signals of a vehicle;

generating a disparity map of the area occupied by at least one object from the final estimate of the three-dimensional shape; and

processing the disparity map with at least one classification algorithm to produce object class confidence data, said classification algorithm is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

7. The apparatus of claim 6, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area.

8. The apparatus of claim 6, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

9. The apparatus of claim 7, wherein the signals comprise airbag enable and disable signals.

10. The apparatus of claim 6, wherein a means for data fusion is performed on the object class confidence data to produce a detected object estimate.

11. An apparatus for object detection comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images, and an output coupled with the processor for outputting information based on an object estimation, wherein the computer system further comprises:

a receiving module for receiving a stereoscopic pair of images of an area occupied by at least one object;

a pattern region detector for detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

an estimator for generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

a subsequent estimator using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

an iterator for iteratively using the subsequent estimate as the initial estimate in the subsequent estimator to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

an estimate processor for processing the final estimate to provide signals, wherein the signals comprise enable and disable signals of a vehicle;

a map generator for generating a disparity map of the area occupied by at least one object from the final estimate of the three-dimensional shape; and

an object class confidence data generator for processing the disparity map with at least one classification algorithm to produce object class confidence data, said classification algorithm is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

12. The apparatus of claim 11, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area.

13. The apparatus of claim 11, wherein the apparatus further comprises an image capturing device for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

14. The apparatus of claim 12, wherein the signals comprise airbag enable and disable signals.

15. The apparatus of claim 11, wherein a data fuser performs data fusion on the object class confidence data to produce a detected object estimate.

16. A computer program product for object detection encoded on a computer-readable medium, having encoded therein, means for:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the means for using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

processing the final estimate to provide signals, wherein the signals comprise enable and disable signals of a vehicle;

generating a disparity map of the area occupied by at least one object from the final estimate of the three-dimensional shape; and

processing the disparity map with at least one classification algorithm to produce object class confidence data, said classification algorithm is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

17. The computer program product of claim 16, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area.

18. The computer program product of claim 16, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

19. The computer program product of claim 17, wherein the signals comprise airbag enable and disable signals.

20. The computer program product of claim 16, wherein a means for data fusion is performed on the object class confidence data to produce a detected object estimate.