Vision-based occupant classification method and system for controlling airbag deployment in a vehicle restraint system

• US 7,505,841 B2
• Filed: 09/02/2005
• Issued: 03/17/2009
• Est. Priority Date: 09/02/2005
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Eiche, David L., Kong, Hongzhi, Nieto, Victor M., Sun, Qin
• Primary Examiner(s): Keith; Jack W.
• Assistant Examiner(s): Nguyen; Chuong P
• Application Number:
  US11/218,671
• Publication Number:
  US 20070055427A1
• Time in Patent Office: 1,292 Days
• Field of Search: 701/1, 701/36, 701/45, 345 7- 9, 345/419, 345581-582, 345611-615, 382/173, 382/181, 382190-195, 382/199, 382206-217, 382/221, 382224-227, 382260-265, 382/171, 382/155, 382/156, 375/240.29
• US Class Current: 701/45
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A vehicle restraint system has a vision-based occupant classification system for control of airbag deployment during a crash scenario. The classification system utilizes two imaging sensors which together create a stream of paired images received and stored by an occupant classification controller. A computer program product of the controller utilizes the paired images to extract disparity/range features and stereo-vision differential edge density features. Moreover, the controller extracts wavelet features from one of the two paired images. All three features or maps are classified amongst preferably seven classifications by algorithms of the computer program product producing class confidence data fed to a sensor fusion engine of the controller for processing and output of an airbag control signal input into a restraint controller of the vehicle restraint system.

Claims

1. A method of vehicle occupant classification for controlling airbag deployment in a vehicle restraint system comprising the steps of:

receiving a first image from a first sensor of a predetermined region of interest occupied by a vehicle occupant by an occupant-classification controller;

simultaneously receiving a second image from a second sensor of the predetermined region of interest occupied by the vehicle occupant by the occupant-classification controller;

dividing the first and second images into an array of equal sized disparity areas;

computing a first texture matrix and a second texture matrix for the respective first and second images;

calculating a single disparity-map-estimate from the first texture matrix and the second texture matrix;

performing an iterative neighborhood update on the disparity map estimate to achieve a disparity map;

applying a smoothing filter to the disparity map to generate a smooth disparity map;

applying an average disparity threshold map and an adaptive offset map to the smooth disparity map to achieve a binary map;

extracting a refined disparity map from the binary map;

inputting the refined disparity map into a neural network classifier of the occupant-classification controller;

utilizing the binary map to develop a segmented single image for calculation of a wavelet transform;

inputting the wavelet transform into the neural network classifier for computation of an array of class confidences and processing the array of class confidences by a processor of the occupant-classification controller for output of the control signal to the vehicle restraint system;

computing a pixel disparity map from each paired pixel of the first and second images;

converting the pixel disparity map into another binary map;

extracting a stereo differential edge density map from the another binary map;

inputting the stereo differential edge density map into the neural network classifier; and

outputting a control signal for controlling airbag deployment in the vehicle restraint system.

2. The method of vehicle occupant classification set forth in claim 1 wherein the neural network classifier includes a child inside of the at-risk-zone classification for computation.

3. The method of vehicle occupant classification set forth in claim 2 wherein the neural network classifier includes a child outside of the at-risk-zone classification for computation.

4. The method of vehicle occupant classification set forth in claim 1 comprising the further steps of:

computing a mono-vision edge density map from the first image; and

inputting the mono-vision edge density map into the neural network classifier.

5. The method of vehicle occupant classification set forth in claim 1 comprising the further steps of:

establishing a three-dimensional physical boundary in a vehicle;

capturing a plurality of images including the first and second images of the physical boundary under varied conditions;

computing a condition-specific disparity map for each of the varied conditions; and

computing the average disparity threshold map by averaging the condition-specific disparity maps together.

6. The method of vehicle occupant classification set forth in claim 5 comprising the further steps of:

computing a standard deviation for each disparity area;

arranging the standard deviations into the adaptive offset map; and

applying the adaptive offset map to the smoothed disparity map to achieve the binary map.

7. The method of vehicle occupant classification set forth in claim 1 comprising the further step of illuminating the predetermined region of interest with infrared light for imaging.

8. The method of vehicle occupant classification set forth in claim 1 comprising the further steps of calculating the average disparity threshold map by averaging disparity values taken over a period of time, between a plurality of images, and exposed to various driving environments.

9. The method set forth in claim 1 wherein the pixel disparity map is computed by constructing an offset-compensated and intensity normalized stereo differential image from the first and second images.

10. The method set forth in claim 9 wherein the pixel disparity map is converted into the other binary map by suppressing noise in the stereo differential image to yield a final stereo differential image and then extracting stereo differential edge densities from the final stereo differential image to yield the other binary map.

11. The method set forth in claim 10 wherein the stereo differential edge density map is extracted by applying morphological operations on the other binary map to yield a refined binary map, dividing the refined binary map into density areas having a predetermined region of interest, and establishing the stereo differential edge density map as a matrix of all of the density areas.