High-performance sensor fusion architecture

• US 20030204384A1
• Filed: 04/24/2002
• Published: 10/30/2003
• Est. Priority Date: 04/24/2002
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Owechko, Yuri, Medasani, Swarup, Srinivasa, Narayan, Boscolo, Riccardo
• Granted Patent: US 7,715,591 B2
• Time in Patent Office: Days
• US Class Current: 703/1
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/431 Frequency domain transformation; Autocorrelation
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals

Abstract

A vision-based system for automatically detecting the type of object within a specified area, such as the type of occupant within a vehicle is presented. The type of occupant can then be used to determine whether an airbag deployment system should be enabled or not. The system extracts different features, including wavelet features and/or a disparity map from images captured by image sensors. These features are then processed by classification algorithms to produce class confidences for various occupant types. The occupant class confidences are fused and processed to determine occupant type. In a preferred embodiment, image features from image edges, wavelet features, and disparity are used. Various classification algorithms may be implemented to classify the object. Use of the disparity map and/or wavelet features provides greater computational efficiency.

Claims

1. A method of object detection comprising the steps of:

receiving images of an area occupied by at least one object;

extracting image features including wavelet features from the images;

classifying the image features to produce object class confidence data; and

performing data fusion on the object class confidence data to produce a detected object estimate.

2. The method of claim 1, wherein the step of classifying image features comprises processing the image features with at least one classification algorithm.

3. The method of claim 2, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

4. The method of claim 2, wherein the step of extracting image features comprises the step of:

extracting wavelet coefficients of the at least one object occupying an area of the images; and

wherein the step of classifying the image features comprises processing the wavelet coefficients with at least one classification algorithm to produce object class confidence data.

5. The method of claim 4, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

6. The method of claim 5, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

7. The method of claim 6, wherein the signals comprise airbag enable and disable signals.

8. The method of claim 7, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

9. The method of claim 4, wherein the step of extracting image features further comprises the steps of:

detecting edges of the at least one object within the images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the step of classifying the image features comprises processing the edge density map with at least one of the classification algorithms to produce object class confidence data.

10. The method of claim 9, wherein the step of extracting image features further comprises the steps of:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the step of using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities, and wherein the step of classifying the image features comprises processing the disparity map with at least one of the classification algorithms to produce object class confidence data.

11. The method of claim 10, further comprising the steps of:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the step of classifying the image features comprises processing the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

12. The method of claim 11, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

13. The method of claim 12, wherein the signals comprise airbag enable and disable signals.

14. The method of claim 13, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

15. The method of claim 14, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

16. The method of claim 10, further comprising the steps of:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the step of classifying the image features comprises processing the motion density map with at least one of the classification algorithms to produce object class confidence data.

17. The method of claim 16, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

18. The method of claim 17, wherein the signals comprise airbag enable and disable signals.

19. The method of claim 18, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

20. The method of claim 19, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

21. The method of claim 4, wherein the step of extracting image features further comprises the steps of:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the step of using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities; and

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities, and wherein the step of classifying the image features comprises processing the disparity map with at least one of the classification algorithms to produce object class confidence data.

22. The method of claim 21, wherein the step of extracting image features further comprises the steps of:

detecting edges of the at least one object within the images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the step of classifying the image features comprises processing the edge density map with at least one of the classification algorithms to produce object class confidence data.

23. The method of claim 22, further comprising the steps of:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the step of classifying the image features comprises processing the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

24. The method of claim 23, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

25. The method of claim 24, wherein the signals comprise airbag enable and disable signals.

26. The method of claim 25, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

27. The method of claim 26, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

28. The method of claim 22, further comprising the steps of:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the step of classifying the image features comprises processing the motion density map with at least one of the classification algorithms to produce object class confidence data.

29. The method of claim 28, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

30. The method of claim 29, wherein the signals comprise airbag enable and disable signals.

31. The method of claim 30, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

32. The method of claim 31, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

33. The method of claim 1, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

34. The method of claim 33, wherein the signals comprise airbag enable and disable signals.

35. The method of claim 34, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

36. The method of claim 35, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

37. A method of object detection comprising the steps of:

receiving a stereoscopic pair of images of an area occupied by at least one object;

extracting image features from the images, with at least a portion of the image features being extracted by the steps of;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the step of using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities;

classifying the image features to produce object class confidence data, with at least a portion of the classifying being performed by processing the disparity map with at least one classification algorithm to produce object class confidence data; and

performing data fusion on the object class confidence data to produce a detected object estimate.

38. The method of claim 37, wherein the step of classifying image features comprises processing the image features with at least one classification algorithm.

39. The method of claim 38, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

40. The method of claim 37, wherein the step of extracting image features further comprises the steps of:

detecting edges of the at least one object within the pair of images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the step of classifying the image features comprises processing the edge density map with at least one of the classification algorithms to produce object class confidence data.

41. The method of claim 40, further comprising the steps of:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the step of classifying the image features comprises processing the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

42. The method of claim 41, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

43. The method of claim 42, wherein the signals comprise airbag enable and disable signals.

44. The method of claim 43, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

45. The method of claim 44, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

46. The method of claim 40, further comprising the steps of:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the step of classifying the image features comprises processing the motion density map with at least one of the classification algorithms to produce object class confidence data.

47. The method of claim 46, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

48. The method of claim 47, wherein the signals comprise airbag enable and disable signals.

49. The method of claim 48, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

50. The method of claim 49, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

51. The method of claim 37, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a step of processing the detected object estimate to provide signals to vehicle systems.

52. The method of claim 51, wherein the signals comprise airbag enable and disable signals.

53. The method of claim 52, wherein the method further comprises a step of capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

54. The method of claim 53, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

55. A computer program product for object detection, the computer program product comprising means, stored on a computer readable medium, for:

receiving images of an area occupied by at least one object;

extracting image features including wavelet features from the images;

classifying the image features to produce object class confidence data; and

performing data fusion on the object class confidence data to produce a detected object estimate.

56. A computer program product for object detection as set forth in claim 55, wherein the means for classifying image features comprises a means for processing the image features with at least one classification algorithm.

57. A computer program product for object detection as set forth in claim 56, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

58. A computer program product for object detection as set forth in claim 55, wherein the means for extracting image features comprises a means for:

extracting wavelet coefficients of the at least one object in the images; and

wherein the means for classifying the image features comprises processing the wavelet coefficients with at least one classification algorithm to produce object class confidence data.

59. A computer program product for object detection as set forth in claim 58, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

60. A computer program product for object detection as set forth in claim 59, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

61. A computer program product for object detection as set forth in claim 60, wherein the signals comprise airbag enable and disable signals.

62. A computer program product for object detection as set forth in claim 61, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

63. A computer program product for object detection as set forth in claim 58, wherein the means for extracting image features further comprises means for:

detecting edges of the at least one object within the images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the means for classifying the image features processes the edge density map with at least one of the classification algorithms to produce object class confidence data.

64. A computer program product for object detection as set forth in claim 63, wherein the means for extracting image features further comprises means for:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the means for using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities, and wherein the means for classifying the image features processes the disparity map with at least one of the classification algorithms to produce object class confidence data.

65. A computer program product for object detection as set forth in claim 64, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

66. A computer program product for object detection as set forth in claim 65, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

67. A computer program product for object detection as set forth in claim 66, wherein the signals comprise airbag enable and disable signals.

68. A computer program product for object detection as set forth in claim 67, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

69. A computer program product for object detection as set forth in claim 68, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

70. A computer program product for object detection as set forth in claim 63, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data.

71. A computer program product for object detection as set forth in claim 70, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

72. A computer program product for object detection as set forth in claim 71, wherein the signals comprise airbag enable and disable signals.

73. The computer program product of claim 72, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

74. A computer program product for object detection as set forth in claim 73, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

75. A computer program product for object detection as set forth in claim 58, wherein the means for extracting image features further comprises means for:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the means for using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities, and wherein the means for classifying the image features processes the disparity map with at least one of the classification algorithms to produce object class confidence data.

76. A computer program product for object detection as set forth in claim 75, wherein the means for extracting image features further comprises means for:

detecting edges of the at least one object within the images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the means for classifying the image features processes the edge density map with at least one of the classification algorithms to produce object class confidence data.

77. A computer program product for object detection as set forth in claim 76, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

78. A computer program product for object detection as set forth in claim 77, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

79. A computer program product for object detection as set forth in claim 78, wherein the signals comprise airbag enable and disable signals.

80. A computer program product for object detection as set forth in claim 79, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

81. A computer program product for object detection as set forth in claim 80, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

82. A computer program product for object detection as set forth in claim 76, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data.

83. A computer program product for object detection as set forth in claim 82, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

84. A computer program product for object detection as set forth in claim 83, wherein the signals comprise airbag enable and disable signals.

85. A computer program product for object detection as set forth in claim 84, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

86. A computer program product for object detection as set forth in claim 85, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

87. A computer program product for object detection as set forth in claim 55, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

88. A computer program product for object detection as set forth in claim 87, wherein the signals comprise airbag enable and disable signals.

89. A computer program product for object detection as set forth in claim 88, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

90. A computer program product for object detection as set forth in claim 89, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

91. A computer program product for object detection, the computer program product comprising means, stored on a computer readable medium, for:

receiving a stereoscopic pair of images of an area occupied by at least one object;

extracting image features from the images, with at least a portion of the image features being extracted by means for;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the means for using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities;

classifying the image features to produce object class confidence data, with at least a portion of the classifying being performed by processing the disparity map with at least one classification algorithm to produce object class confidence data; and

performing data fusion on the object class confidence data to produce a detected object estimate.

92. A computer program product for object detection as set forth in claim 91, wherein the means for classifying image features processes the image features with at least one classification algorithm.

93. A computer program product for object detection as set forth in claim 92, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

94. A computer program product for object detection as set forth in claim 91, wherein the means for extracting image features further comprises means for:

detecting edges of the at least one object within the images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the means for classifying the image features processes the edge density map with at least one of the classification algorithms to produce object class confidence data.

95. A computer program product for object detection as set forth in claim 94, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

96. A computer program product for object detection as set forth in claim 95, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

97. A computer program product for object detection as set forth in claim 96, wherein the signals comprise airbag enable and disable signals.

98. A computer program product for object detection as set forth in claim 97, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

99. A computer program product for object detection as set forth in claim 98, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

100. A computer program product for object detection as set forth in claim 91, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data.

101. A computer program product for object detection as set forth in claim 100, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

102. A computer program product for object detection as set forth in claim 101, wherein the signals comprise airbag enable and disable signals.

103. A computer program product for object detection as set forth in claim 102, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

104. A computer program product for object detection as set forth in claim 103, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

105. A computer program product for object detection as set forth in claim 91, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

106. A computer program product for object detection as set forth in claim 105, wherein the signals comprise airbag enable and disable signals.

107. A computer program product for object detection as set forth in claim 106, wherein the computer program product further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

108. A computer program product for object detection as set forth in claim 107, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

109. An apparatus for object detection comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images, and an output coupled with the processor for outputting information based on an object estimation, wherein the computer system further comprises means, residing in its processor and memory, for:

receiving images of an area occupied by at least one object;

extracting image features including wavelet features from the images;

classifying the image features to produce object class confidence data; and

performing data fusion on the object class confidence data to produce a detected object estimate.

110. An apparatus for object detection as set forth in claim 109, wherein the means for classifying image features comprises a means for processing the image features with at least one classification algorithm.

111. An apparatus for object detection as set forth in claim 110, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

112. An apparatus for object detection as set forth in claim 109, wherein means for extracting image features comprises a means for:

extracting wavelet coefficients of the at least one object in the images; and

wherein the means for classifying the image features comprises processing the wavelet coefficients with at least one classification algorithm to produce object class confidence data.

113. An apparatus for object detection as set forth in claim 112, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

114. An apparatus for object detection as set forth in claim 113, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

115. An apparatus for object detection as set forth in claim 114, wherein the signals comprise airbag enable and disable signals.

116. An apparatus for object detection as set forth in claim 115, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

117. An apparatus for object detection as set forth in claim 112, wherein the means for extracting image features further comprises means for:

detecting edges of the at least one object within the images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the means for classifying the image features processes the edge density map with at least one of the classification algorithms to produce object class confidence data.

118. An apparatus for object detection as set forth in claim 117, wherein the means for extracting image features further comprises means for:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the means for using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities, and wherein the means for classifying the image features processes the disparity map with at least one of the classification algorithms to produce object class confidence data.

119. An apparatus for object detection as set forth in claim 118, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

120. An apparatus for object detection as set forth in claim 119, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

121. An apparatus for object detection as set forth in claim 120, wherein the signals comprise airbag enable and disable signals.

122. An apparatus for object detection as set forth in claim 121, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

123. An apparatus for object detection as set forth in claim 122, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

124. An apparatus for object detection as set forth in claim 118, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data.

125. An apparatus for object detection as set forth in claim 124, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

126. An apparatus for object detection as set forth in claim 125, wherein the signals comprise airbag enable and disable signals.

127. An apparatus for object detection as set forth in claim 126, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

128. An apparatus for object detection as set forth in claim 127, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

129. An apparatus for object detection as set forth in claim 112, wherein the means for extracting image features further comprises means for:

receiving a stereoscopic pair of images of an area occupied by at least one object;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the means for using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities, and wherein the means for classifying the image features processes the disparity map with at least one of the classification algorithms to produce object class confidence data.

130. An apparatus for object detection as set forth in claim 129, wherein the means for extracting image features further comprises means for:

detecting edges of the at least one object within the images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the means for classifying the image features processes the edge density map with at least one of the classification algorithms to produce object class confidence data.

131. An apparatus for object detection as set forth in claim 130, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

132. An apparatus for object detection as set forth in claim 131, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

133. An apparatus for object detection as set forth in claim 132, wherein the signals comprise airbag enable and disable signals.

134. An apparatus for object detection as set forth in claim 133, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

135. An apparatus for object detection as set forth in claim 134, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

136. An apparatus for object detection as set forth in claim 130, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data.

137. An apparatus for object detection as set forth in claim 136, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

138. An apparatus for object detection as set forth in claim 137, wherein the signals comprise airbag enable and disable signals.

139. An apparatus for object detection as set forth in claim 138, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

140. An apparatus for object detection as set forth in claim 139, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

141. An apparatus for object detection as set forth in claim 109, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

142. An apparatus for object detection as set forth in claim 141, wherein the signals comprise airbag enable and disable signals.

143. An apparatus for object detection as set forth in claim 142, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

144. An apparatus for object detection as set forth in claim 143, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

145. An apparatus for object detection comprising a computer system including a processor, a memory coupled with the processor, an input coupled with the processor for receiving images, and an output coupled with the processor for outputting information based on an object estimation, wherein the computer system further comprises means, residing in its processor and memory, for:

receiving a stereoscopic pair of images of an area occupied by at least one object;

extracting image features from the images, with at least a portion of the image features being extracted by means for;

detecting pattern regions and non-pattern regions within each of the pair of images using a texture filter;

generating an initial estimate of spatial disparities between the pattern regions within each of the pair of images;

using the initial estimate to generate a subsequent estimate of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using disparity constraints;

iteratively using the subsequent estimate as the initial estimate in the means for using the initial estimate to generate a subsequent estimate in order to generate further subsequent estimates of the spatial disparities between the non-pattern regions based on the spatial disparities between the pattern regions using the disparity constraints until there is no change between the results of subsequent iterations, thereby generating a final estimate of the spatial disparities;

generating a disparity map of the area occupied by at least one object from the final estimate of the spatial disparities;

classifying the image features to produce object class confidence data, with at least a portion of the classifying being performed by processing the disparity map with at least one classification algorithm to produce object class confidence data; and

performing data fusion on the object class confidence data to produce a detected object estimate.

146. An apparatus for object detection as set forth in claim 145, wherein the means for classifying image features processes the image features with at least one classification algorithm.

147. An apparatus for object detection as set forth in claim 146, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

148. An apparatus for object detection as set forth in claim 145, wherein the means for extracting image features further comprises means for:

detecting edges of the at least one object within the images;

masking the edges with a background mask to find important edges;

calculating edge pixels from the important edges; and

producing edge density maps from the important edges, the edge density map providing the image features, and wherein the means for classifying the image features processes the edge density map with at least one of the classification algorithms to produce object class confidence data.

149. An apparatus for object detection as set forth in claim 148, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data, which is used independently of the data fusion to produce an independent detected object estimate.

150. An apparatus for object detection as set forth in claim 149, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

151. An apparatus for object detection as set forth in claim 150, wherein the signals comprise airbag enable and disable signals.

152. An apparatus for object detection as set forth in claim 151, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

153. An apparatus for object detection as set forth in claim 152, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

154. An apparatus for object detection as set forth in claim 148, further comprising means for:

detecting motion of the at least one object within the images;

calculating motion pixels from the motion; and

producing motion density maps from the motion pixels, the motion density map providing the image features; and

wherein the means for classifying the image features processes the motion density map with at least one of the classification algorithms to produce object class confidence data.

155. An apparatus for object detection as set forth in claim 154, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

156. An apparatus for object detection as set forth in claim 155, wherein the signals comprise airbag enable and disable signals.

157. An apparatus for object detection as set forth in claim 156, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

158. An apparatus for object detection as set forth in claim 157, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.

159. An apparatus for object detection as set forth in claim 145, where the at least one object comprises a vehicle occupant and the area comprises a vehicle occupancy area, and further comprising a means for processing the detected object estimate to provide signals to vehicle systems.

160. An apparatus for object detection as set forth in claim 159, wherein the signals comprise airbag enable and disable signals.

161. An apparatus for object detection as set forth in claim 160, wherein the apparatus further comprises a means for capturing images from a sensor selected from a group consisting of CMOS vision sensors and CCD vision sensors.

162. An apparatus for object detection as set forth in claim 161, wherein at least one of the classification algorithms is selected from the group consisting of a trained C5 decision tree, a trained Nonlinear Discriminant Analysis network, and a trained Fuzzy Aggregation Network.