Method and apparatus for controlling classification and classification switching in a vision system

• US 7,636,479 B2
• Filed: 02/24/2004
• Issued: 12/22/2009
• Est. Priority Date: 02/24/2004
• Status: Active Grant

• Current Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Original Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Inventors: Khairallah, Farid, Luo, Yun, Wallace, Jon K.
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US10/785,920
• Publication Number:
  US 20050185846A1
• Time in Patent Office: 2,128 Days
• Field of Search: 177/144, 180/271, 180/273, 180/282, 280/734, 280/735, 382/103, 382/104, 382/224, 701/45, 701/46, 701/49
• US Class Current: 382/224
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/262 using context analysis, e.g. lexical, syntactic or semantic context
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A system for classifying image data includes a vision system (58) for continuously imaging an area (42) of interest and providing an image signals. A classifier (84) determines an image class corresponding to each image signal, determines a confidence level for each determined image class, determines a time period for the confidence level, and establishes an image classification in response thereto. Switching between image classifications is in response to the confidence level and time period of the confidence level. When used in a vehicle restraining system (20), other monitor vehicle parameters (54) can be used to adjust the confidence levels and time periods needed to switch between image classifications.

Claims

1. An occupant classification method for controlling an occupant protection device in a vehicle comprising the steps of:

continuously imaging an area of interest within an interior of the vehicle at an associated imaging sensor and providing image signals;

determining an image class corresponding to each image signal from a plurality of image classes;

determining, for a predetermined time period, respective confidence levels for the plurality of image classes, the confidence level for a given class being defined as the percentage of image class determinations that match the image class out of a total number of image class determinations made during the predetermined time period;

establishing an image classification in response to the determined confidence levels for the plurality of image classes, only if the confidence level associated with a given class over the predetermined time period exceeds a threshold value; and

controlling the occupant protection device according to the established image classification.

2. The method of claim 1 further comprising the steps of:

further monitoring the image signals;

determining if a further monitored image signal indicates a different class;

switching the established image classification to a new established image classification when a confidence level associated with an image class exceeds a switching confidence threshold for a switching time period.

3. The method of claim 1 wherein the system is used in a vehicle restraining system and further comprises the steps of:

monitoring vehicle input signals;

adjusting the switching time period and switching confidence threshold in response to said monitored vehicle input signals.

4. The method of claim 3 wherein said step of monitoring vehicle input signals includes monitoring an open condition of a vehicle door.

5. The method of claim 3 wherein said step of monitoring vehicle input signals includes monitoring a condition of the vehicle ignition.

6. The method of claim 1 wherein the step of imaging includes using a stereo camera.

7. The method of claim 1 wherein the step of imaging further includes the step of removing background information and noise from the image signal.

8. The method of claim 7 wherein the step of removing background information and noise includes applying a contrast limited adaptive histogram equalization that adjusts the image for lighting conditions.

9. The method of claim 1 wherein the step of determining an image class uses at least one of a support vector machine and a neural network.

10. A method for controlling an occupant protection device in a vehicle according to classification of image data comprising the steps of:

imaging an area of interest within an interior of the vehicle at an image sensor and providing image data;

classifying the image data as one of a plurality of image classes to provide an image classification;

storing a predetermined number of image classifications over a first time period;

assigning a confidence level to each image class according to the percentage of the predetermined number of image classifications in the first time period that represent the image class;

assigning an image class to the area of interest only when the confidence level assigned to the image class exceeds a first threshold; and

controlling the occupant protection device according to the established image classification.

11. The method of claim 10, further comprising switching the assigned image class for the area of interest when a confidence level greater than a second threshold is assigned to an image class, the second threshold being greater than the first threshold.

12. A system for classifying image data comprising:

a vision system for continuously imaging an area of interest and providing image signals;

a classifier that determines an image class corresponding to each image signal; and

a confidence filter that determines a confidence level for each determined image class, the confidence filter determining a first confidence level for a first image class over a first time period and determining a second confidence level for a second image class over a second time period, and establishing an image classification for the area of interest as the first class if the first confidence level exceeds a first threshold and as the second class if the second confidence level exceeds a second threshold.

13. The system of claim 12, wherein the first time period is shorter than the second time period and the first confidence value is greater than the second confidence value.

14. A system for classifying image data associated with a vehicle occupant safety system into an associated one of a plurality of output classes, said system comprising:

a vision system that images a vehicle interior to provide an image signal; and

a classifier that determines an image class in response to a class confidence level of the image signal and time period of the classification, said classifier permitting class switching only in response to a new image class determination having a predetermined new confidence value for a predetermined time period.

15. The system of claim 14 wherein the vision system comprises a stereo vision system that produces three-dimension image data of the vehicle interior as a stereo disparity map.