Image processing system for dynamic suppression of airbags using multiple model likelihoods to infer three dimensional information

• US 20030016845A1
• Filed: 07/10/2001
• Published: 01/23/2003
• Est. Priority Date: 07/10/2001
• Status: Active Grant

• Current Assignee: Eaton Corp. (Wisconsin) (Eaton Corporation PLC.)
• Original Assignee: Eaton Corp. (Wisconsin) (Eaton Corporation PLC.)
• Inventors: Farmer, Michael
• Granted Patent: US 6,925,193 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60R 2021/0044 Chest
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01542 detecting passenger motion
• CPC Class Codes: B60R 21/01552 detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

The present invention relates in general to systems used to determine whether airbag deployment should be disabled or modified due to the proximity of the occupant to the airbag. In particular, the present invention is an image processing system that uses an image and signal process that utilizes real-time streaming video-images from a video camera to determine if the occupant is too close to the air bag, or will be to close the air bag by the time that the airbag deploys. In a crash situation, the system quickly determines whether or not the airbag should be disabled or modified. The process uses a multiple-model Kalman filter to infer three-dimensional information from a sequence of two-dimensional images. The system predicts the position and shape of the occupant at a faster rate than the rate in which the camera collects data.

Claims

1. An image processing system for use with an airbag deployment system having a seat, an occupant in the seat, a sensor for generating sensor measurements, an airbag, and an airbag controller, said image processing system comprising:

an iterative tracking and predicting system, wherein said tracking and predicting system tracks and predicts the movement of an occupant in a seat by incorporating the most recent sensor measurement and past predictions into an estimate of the occupant'"'"'s position.

2. An image processing system as in claim 1, which can generate predictions at a quicker rate than the rate at which the sensor captures measurements.

3. An image processing system as in claim 1, wherein said iterative tracking and predicting system utilizes a plurality of Kalman filters.

4. An image processing system as in claim 1, further comprising an at-risk-zone intrusion indicator, wherein said at-risk-zone intrusion indicator uses said prediction from said iterative tracking and predicting system to determine whether an occupant would be in a predefined at-risk-zone by the time an airbag were to deploy.

5. An image processing system as in claim 1, said tracking and predicting system further comprising:

a shape tracker and predictor subsystem, wherein said shape tracker and predictor subsystem tracks and predicts characteristics indicating the shape of the occupant; and

a motion tracker and predictor subsystem, wherein said motion tracker and predictor subsystem tracks and predicts characteristics indicating the motion of the occupant.

6. An image processing system as in claim 5, wherein the characteristics tracked and predicted by said shape tracker and predictor subsystem comprise:

a height of the occupant; and

a shape of the occupant'"'"'s upper torso.

7. An image processing system as in claim 5, wherein said characteristics tracked and predicted by said motion tracker and predictor subsystem comprise:

a distance between the occupant and the airbag; and

a forward tilt angle.

8. An image processing system as in claim 5, wherein said shape tracker and predictor utilizes predefined shape states.

9. An image processing system as in claim 8, wherein said predefined states comprise

a state of leaning left;

a state of leaning right; and

a state of sitting generally centered.

10. An image processing system as in claim 5, wherein said motion tracker and predictor utilizes predefined motion modes.

11. An image processing system as in claim 10, wherein said predefined motion modes comprise:

a mode of crashing;

a mode of being stationary; and

a mode of being human.

12. An image classification system as in claim 5, wherein said shape tracker and predictor comprises:

an update shape predictor;

an update covariance and gain matrices generator;

an update shape estimator; and

a combined shape estimate generator.

13. An image classification system as in claim 5, wherein said motion tracker and predictor comprises:

an update motion predictor;

an update covariance and gain matrices generator;

an update motion estimator; and

a combined motion estimate generator.

14. An image processing system as in claim 5, wherein said shape tracker and predictor determines a sideways tilt angle of the occupant.

16. An image processing system as in claim 15, wherein said tracking and predicting system utilizes an upper ellipse to represent the upper torso of the occupant.

17. An image processing system as in claim 15, wherein said tracking and predicting system utilizes a plurality of Kalman filters for various image characteristics.

18. An image processing system as in claim 15, wherein said tracking and predicting system can generate estimates on a more frequent basis that the sensor can capture occupant images.

19. An image processing system as in claim 15, wherein said tracking and predicting system further comprises:

a shape tracker and predictor subsystem, wherein said shape tracker and predictor subsystem tracks and predicts characteristics indicating the shape of the occupant; and

a motion tracker and predictor subsystem, wherein said motion tracker and predictor subsystem tracks and predicts characteristics indicating the motion of the occupant image with respect to the location of the airbag.

15. An image processing system for use with an airbag deployment system having a seat, an occupant in the seat, an optical sensor, an airbag, a segmentation device, an airbag controller, said image processing system comprising:

a tracking and predicting system, wherein said tracking and predicting system tracks a plurality of variables regarding an occupant;

an ARZ intrusion indicator, wherein said ARZ intrusion indicator determines which point of the occupant'"'"'s upper torso is closest to the airbag, and whether the occupant will be in a position within a predefined danger zone by the time the airbag could be deployed; and

wherein said tracking and predicting system integrates past predictions with current measurements utilizing probability weighted heuristics.

20. An image processing system comprising:

a sensor for capturing an image of a seat area;

a segmentation subsystem for extracting a segmented image of an occupant from an ambient image of a seat area;

an ellipse fitting subsystem which fits an upper ellipse to the upper torso of an occupant, a lower ellipse to the bottom portion of an occupant, and a centroid at a point of intersection between said upper ellipse and said lower ellipse;

a tracking and predicting system, further comprising a shape tracker and predictor and a motion tracker and predictor; and

an at-risk-zone (“

ARZ”

) intrusion indicator, wherein said ARZ intrusion indicator determines which point on the upper ellipse is closest to the airbag, and wherein said ARZ intrusion indicator informs the airbag controller whether the occupant will be in a position within a predefined danger zone by the time the airbag is deployed;

wherein said shape tracker and predictor tracks and predicts predefined shape characteristics using a Kalman filter equation for each predefined shape state, and wherein said shape tracker and predictor weighs each Kalman filter using the predefined probabilities associated with said shape states, to generate an overall prediction for each said shape characteristic; and

wherein said motion tracker and predictor predicts predefined motion characteristics using a Kalman filer equation for each predefined motion mode and weighs the predefined probabilities associated with said motion modes, to generate an overall prediction for each said motion characteristic.

21. A method for predicting the movement of an image, comprising the steps of:

applying a plurality of mathematical heuristics to a plurality of image characteristics to incorporate past measurements and past predictions into an updated overall prediction of said plurality of image characteristics, wherein the image is in one of a plurality of predefined modes and states; and

using said updated characteristic predictions to determine whether the image will enter an ARZ;

wherein and motion characteristics.