Image processing system for detecting when an airbag should be deployed

• US 6,662,093 B2
• Filed: 01/17/2002
• Issued: 12/09/2003
• Est. Priority Date: 05/30/2001
• Status: Expired due to Fees

• Current Assignee: Eaton Corp. (Wisconsin) (Eaton Corporation PLC.)
• Original Assignee: Eaton Corp. (Wisconsin) (Eaton Corporation PLC.)
• Inventors: Farmer, Michael Edward
• Primary Examiner(s): Cuchlinski, Jr., William A.
• Assistant Examiner(s): Marc, McDieunel, Marc, McDieunel
• Application Number:
  US10/052,152
• Publication Number:
  US 20030040859A1
• Time in Patent Office: 691 Days
• Field of Search: 701/28, 701/45, 701/47, 180/268, 180/271, 280/735, 340/438, 382/103, 382/104, 382/106
• US Class Current: 701/45
• CPC Class Codes: B60R 2021/0044 Chest
• CPC Class Codes: B60R 2021/01315 monitoring occupant displacement
• CPC Class Codes: B60R 21/013 including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01542 detecting passenger motion
• CPC Class Codes: B60R 21/01552 detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: B60R 21/01556 Child-seat detection systems
• CPC Class Codes: G06F 18/24765 Rule-based classification
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/765 using rules for classification or partitioning the feature space
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

The invention can determine whether a collision has occurred, whether an airbag should be deployed, and what the desirable strength of an airbag deployment should be based on occupant characteristics obtained from the segmented image of the occupant. Using interacting multiple model Kalman filters, the invention can infer three-dimensional characteristics from a two-dimensional image, and predict the location, velocity, and acceleration of the occupant. Kalman filters are used with respect to all measurements to incorporate past predictions and measurements into the most recent estimates and predictions in order to eliminate the “noise” associated with any particular measurement. The system can predict the position and shape of the occupant at a faster rate than the rate at which the sensor collects data. Occupant characteristics can be compared to corresponding predetermined data associated with various occupant modes, including the mode of crashing, to determine if a collision has occurred.

Claims

1. An image processing system for use with an airbag deployment system having a seat, an occupant in the seat, a sensor for generating sensor readings, an airbag, and an airbag controller, said image processing system comprising:

a segmented image of the occupant obtained from the sensor reading;

a tracking and predicting subsystem, including an occupant characteristic prediction, said tracking and predicting subsystem identifying said occupant characteristic prediction at least in part from said segmented image; and

a crash detection subsystem, including a crash determination, said crash detection subsystem generating said crash determination from said occupant characteristic prediction.

2. An image processing system as in claim 1, wherein said occupant characteristic prediction is an acceleration of the occupant.

3. An image processing system as in claim 2, wherein said acceleration of the occupant characteristic prediction is a future acceleration of the occupant.

4. An image processing system as in claim 2, wherein said acceleration of the occupant is a probability-weighted acceleration of the occupant.

5. An image processing system as in claim 1, wherein said crash determination is generated at a quicker rate then the rate at which the sensor generates sensor readings.

6. An image processing system as in claim 1, further comprising an ellipse fitting subsystem, an ellipse, and an ellipse characteristic, said ellipse fitting subsystem generating said ellipse from said segmented image, and wherein said occupant characteristic prediction is said ellipse characteristic.

7. An image processing system as in claim 1, said tracking and predicting subsystem further including an iterative tracking and predicting heuristic for generating said occupant characteristic prediction.

8. An image processing system as in claim 7, said iterative tracking and predicting heuristic comprising a Kalman filter, said Kalman filter generating said occupant characteristic prediction.

9. An image processing system as in claim 1, further comprising a plurality of motion models and a probability-weighted calculation, wherein said crash determination is generated by performing said probability-weighted calculation on said occupant characteristic for each motion model in said plurality of motion models.

10. An image processing system as in claim 1, further comprising an impact assessment subsystem and an impact metric, said impact assessment subsystem generating said impact metric from said occupant characteristic prediction.

11. An image processing system as in claim 10, wherein a kinetic energy prediction is said impact metric.

12. An image processing system as in claim 11, wherein said impact metric relates to a point in time when the occupant impacts into the deploying airbag.

13. An image processing system as in claim 10, further comprising an airbag deployment subsystem and a deployment strength, said airbag deployment subsystem determining said deployment strength from said impact metric.

14. An image processing system as in claim 13, further comprising a seat belt status, said airbag deployment subsystem incorporating said seat belt status into the determination of said deployment strength.

15. An image processing system as in claim 1, further including a plurality of occupant characteristics comprising:

a location; and

one said occupant characteristic selected from the group including;

a velocity;

an acceleration;

a mass;

a volume;

an area; and

an axis of a three-dimensional environment.

35. An image processing system as in claim 1, wherein said tracking and predicting subsystem identifies said occupant characteristic prediction from said segmented image and a prior occupant characteristic prediction.

16. An image processing system for use with an airbag deployment system having a seat, an occupant in the seat, a sensor for generating sensor readings, an airbag, and an airbag controller, said image processing system comprising:

a segmented image of the occupant obtained from the sensor reading;

a tracking and predicting subsystem, including an occupant characteristic, said tracking and predicting subsystem identifying said occupant characteristic from said segmented image, said tracking and predicting subsystem further including;

a plurality of occupant characteristics, comprising a plurality of shape characteristics and a plurality of motion characteristics,

a motion tracker and predictor for tracking and predicting said plurality of motion characteristics, and

a shape tracker and predictor for tracking and predicting said plurality of shape characteristics, wherein said occupant characteristic is generated from at least one said motion characteristic and at least one said shape characteristic; and

a crash detection subsystem, including a crash determination, said crash detection subsystem generating said crash determination from said occupant characteristic.

17. An image processing system as in claim 16, wherein said shape characteristics include:

a height of the occupant; and

a shape of the upper torso of the occupant.

18. An image processing system as in claim 17, including a plurality of predefined motion modes, wherein said motion tracker and predictor utilizes said plurality of predefined motion modes.

19. An image processing system as in claim 18, wherein said predefined motion modes comprise:

a mode of crashing;

a mode of pre-crash braking;

a mode of being stationary; and

a mode of being human.

20. An image processing system as in claim 16, wherein said motion characteristics include:

a distance between the occupant and the airbag; and

a forward tilt angle.

21. An image processing system as in claim 16, including a plurality of predefined shape states, wherein said shape tracker and predictor uses said plurality of predefined shape states.

22. An image processing system as in claim 21, wherein said predefined shape states comprise:

a state of leaning left;

a state of leaning right; and

a state of sitting generally centered.

23. An image classification system as in claim 16, wherein said shape tracker and predictor comprises:

an update shape predictor;

an update covariance and gain matrices generator;

an update shape estimator; and

a combined shape estimate generator.

24. An image classification system as in claim 16, wherein said motion tracker and predictor comprises:

an update motion predictor;

an update covariance and gain matrices generator;

an update motion estimator; and

a combined motion estimate generator.

25. An image processing system for use with an airbag deployment system having a seat, an occupant in the seat, a sensor for capturing ambient images, an airbag, an airbag controller, said image processing system comprising:

a segmentation subsystem, including an ambient image and a segmented image, said segmentation subsystem generating said segmented image from said ambient image;

an ellipse fitting subsystem, including an ellipse, said ellipse fitting subsystem generating said ellipse from said segmented image;

a tracking and predicting subsystem, including a plurality of occupant characteristics, said tracking an predicting subsystem generating said plurality of occupant characteristics from said ellipse; and

a crash detection subsystem, including a crash determination, said crash detection subsystem generating said crash determination from said plurality of occupant characteristics.

26. An image processing system as in claim 25, further comprising a plurality of motion models and a probability-weighted calculation, wherein said crash determination is generated by performing said probability-weighted calculation on said plurality of occupant characteristics for each motion model in said plurality of motion models.

27. An image processing system as in claim 25, said tracking and predicting subsystem further including a plurality of past predictions, wherein said plurality of past predictions are incorporated into said plurality of occupant characteristics.

28. An image processing system as recited in claim 27, said tracking and predicting subsystem applying a plurality of Kalman filters to incorporate said plurality of past predictions into said plurality of occupant characteristics.

29. A method for detecting a vehicle collision by evaluating characteristics of the occupant in the vehicle, comprising the steps of:

isolating a segmented image of the occupant;

applying a plurality of mathematical heuristics to the segmented image of the occupant to generate a plurality of characteristics relating to the occupant; and

determining which of a plurality of motion models best describes said plurality of characteristics.

30. A method for detecting a vehicle collision as in claim 29, further comprising creating an ellipse to represent the upper torso of the segmented image, wherein said plurality of characteristics are derived from said ellipse.

31. A method for detecting a vehicle collision as in claim 29, wherein said plurality of motion models include at least a subset of a stationary mode, a human mode, a pre-crash braking mode, and a crash mode.

32. A method for detecting a vehicle collision as in claim 29, wherein determining which of a plurality of motion models best describes said plurality of occupant characteristics comprises comparing at least one said occupant characteristic with a corresponding motion model characteristic.

33. A method for detecting a vehicle collision as in claim 29, wherein said plurality of mathematical heuristics includes a Kalman filter.

34. A method for detecting a vehicle collision as in claim 33, wherein said Kalman filter is a interacting multiple model implementation Kalman filter.