Vision-based occupant classification method and system for controlling airbag deployment in a vehicle restraint system

• US 20070055427A1
• Filed: 09/02/2005
• Published: 03/08/2007
• Est. Priority Date: 09/02/2005
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG (Aptiv PLC)
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Nieto, Victor, Sun, Qin, Kong, Hongzhi, Eiche, David
• Granted Patent: US 7,505,841 B2
• Time in Patent Office: Days
• US Class Current: 701/45
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A vehicle restraint system has a vision-based occupant classification system for control of airbag deployment during a crash scenario. The classification system utilizes two imaging sensors which together create a stream of paired images received and stored by an occupant classification controller. A computer program product of the controller utilizes the paired images to extract disparity/range features and stereo-vision differential edge density features. Moreover, the controller extracts wavelet features from one of the two paired images. All three features or maps are classified amongst preferably seven classifications by algorithms of the computer program product producing class confidence data fed to a sensor fusion engine of the controller for processing and output of an airbag control signal input into a restraint controller of the vehicle restraint system.

Claims

1. A method of vehicle occupant classification for controlling airbag deployment in a vehicle restraint system comprising the steps of:

receiving a first image from a first sensor of a predetermined region of interest occupied by a vehicle occupant by an occupant-classification controller;

simultaneously receiving a second image from a second sensor of the predetermined region of interest occupied by the vehicle passenger by the occupant-classification controller;

dividing the first and second images into an array of equal sized disparity areas;

computing a first texture matrix and a second texture matrix for the respective first and second images;

calculating a single disparity-map-estimate from the first texture matrix and the second texture matrix;

performing an iterative neighborhood update to achieve a disparity map;

applying a pre-determined average disparity threshold map to the disparity map to achieve a binary map;

extracting a refined disparity map from the binary map;

inputting the refined disparity map into a neural network classifier of the occupant-classification controller; and

outputting a control signal for controlling airbag deployment in the vehicle restraint system.

2. The method of vehicle occupant classification set forth in claim 1 comprising the further step of applying a smoothing filter after performing the iterative neighborhood update and before applying the average disparity threshold map for enhancing the disparity map into a smooth disparity map.

3. The method of vehicle occupant classification set forth in claim 1 comprising the further step of utilizing the binary map to develop a segmented single image for calculation of a wavelet transform.

4. The method of vehicle occupant classification set forth in claim 3 comprising the further step of inputting the wavelet transform into the neural network classifier for computation of an array of class confidences and processing the array of class confidences by a processor of the occupant-classification controller for output of the control signal to the vehicle restraint system.

5. The method of vehicle occupant classification set forth in claim 4 comprising the further steps of:

computing a pixel disparity map from each paired pixel of the first and second images;

convert the pixel disparity map into a binary map;

extract stereo differential edge density map from the binary map; and

inputting the stereo differential edge density map into the neural network classifier.

6. The method of vehicle occupant classification set forth in claim 5 wherein the neural network classifier includes a child inside of the at-risk-zone classification for computation.

7. The method of vehicle occupant classification set forth in claim 6 wherein the neural network classifier includes a child outside of the at-risk-zone classification for computation.

8. The method of vehicle occupant classification set forth in claim 4 comprising the further steps of:

computing a mono-vision edge density map from the first image; and

inputting the mono-vision edge density map into the neural network classifier.

9. The method of vehicle occupant classification set forth in claim 1 comprising the further steps of:

establishing a three-dimensional physical boundary in a vehicle;

capturing a plurality of images of the physical boundary under varied conditions;

computing a condition-specific disparity map for each of the varied conditions; and

computing the average disparity threshold map by averaging the condition-specific disparity maps together.

10. The method of vehicle occupant classification set forth in claim 1 comprising the further step of applying a pre-determined adaptive offset map to the disparity map to achieve the binary map.

11. The method of vehicle occupant classification set forth in claim 9 comprising the further steps of:

computing a standard deviation for each disparity area;

arranging the standard deviations into an adaptive offset map; and

applying the adaptive offset map to the disparity map to achieve the binary map.

12. The method of vehicle occupant classification set forth in claim 1 comprising the further step of illuminating the region of interest with infrared light for imaging.

13. The method of vehicle occupant classification set forth in claim 1 comprising the further steps of calculating the disparity threshold map by averaging disparity values taken over a period of time, between a plurality of images, and exposed to various driving environments.

14. A vision-based occupant classification system for controlling airbag deployment in a vehicle restraint system having a computer-based processor, at least one accelerometer for detection of a crash scenario by inputting a signal to the processor, and an airbag for controlled deployment during the crash scenario and being controlled by a restraint controller, the vision-based occupant classification system comprising:

a first and a second imaging sensor for simultaneous imaging of a predetermined region of interest;

a first image and a second image produced by the respective first and second imaging sensors;

a computer-based occupant-classification controller for classification of the pair of images, the occupant-classification controller having a memory;

a computer program software medium encoded into the occupant-classification controller;

wavelet features extracted from the first image;

disparity features extracted from the first and second images;

differential edge density features extracted from the first and second images; and

a neural network classifier of the computer program software medium for computation of the wavelet, disparity and differential edge density features processed by the computer program software medium for processing of an array of class confidence data into an airbag control signal received by the vehicle restraint controller.

15. The vision-based occupant classification system set forth in claim 14 comprising an infrared light for illuminating the pre-determined region of interest.

16. The vision-based occupant classification system set forth in claim 14 comprising:

a pre-determined segmentation template stored in the memory and utilized by the computer program software medium;

an area disparity map of the first and second images computed by the computer program software medium;

a binary map computed by the computer program software medium from the segmentation template and the area disparity map; and

a segmented disparity map having the disparity features and computed by the computer program software medium from the binary map and the area disparity map.

17. The vision-based occupant classification system set forth in claim 16 comprising a segmented single image of the first image computed by the computer program software medium by use of the segmentation template, and wherein the wavelet features are extracted from the segmented single image.