Application of human facial features recognition to automobile safety

• US 6,724,920 B1
• Filed: 07/21/2000
• Issued: 04/20/2004
• Est. Priority Date: 07/21/2000
• Status: Expired due to Fees

• Current Assignee: TRW Limited (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Original Assignee: TRW Limited (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Inventors: Shreve, Gregory A., Niesen, Joseph W., Dunbridge, Barry, McIver, George W., Berenz, John J.
• Primary Examiner(s): Patel, Jayanti K.
• Assistant Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US09/621,160
• Time in Patent Office: 1,369 Days
• Field of Search: 382/105, 382/103, 382/104, 382/106, 382/118, 382/278, 280/735, 280/730.1, 280/734, 340/436, 340/426, 340/438, 340/439, 340/988, 340/573.1, 340/575, 340/576, 340/5.53, 340/425.5, 701/45, 701/49, 701/46, 250/338.5, 307/10.1, 180/169, 180/272
• US Class Current: 382/118
• CPC Class Codes: B60R 21/01534 using electromagneticwaves, e.g. infrared
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01542 detecting passenger motion
• CPC Class Codes: B60R 21/01552 detecting position of specific human body parts, e.g. face, eyes or hands
• CPC Class Codes: B60S 1/0862 including additional sensors vehicle speed sensors B60S1/0896
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19647 Systems specially adapted for intrusion detection in or around a vehicle
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

An imaging system (50) for providing vehicle safety features that employs face recognition software to identify and track a person. The system (50) employs infrared emitters (30) that emit an infrared signal along a predetermined field-of-view, and an infrared sensor (34), such as a CMOS sensor used as a video signal array, that receives reflected infrared illumination from objects in the field-of-view. A processor (52) including the face recognition software, is employed to detect human faces to identify and track the person. Once a face is detected, it can be compared to a data base to identify the person. Various applications for the imaging system (50) for providing vehicle safety features include identifying the driver or passenger for personalizing the vehicle'"'"'s airbags, providing pre-crash collision avoidance, providing blind spot detection, providing vehicle crash recording, and providing a warning signal if the driver appears drowsy.

Claims

1. An imaging system for a vehicle, said system comprising:

an infrared source generating an infrared beam along a particular field-of-view in or around the vehicle;

an infrared detector responsive to infrared radiation reflected from objects in the field-of-view of the infrared beam, said detector generating image signals from the reflected infrared radiation; and

a processor responsive to the image signals from the detector, said processor including face recognition software that images and identifies a plurality of separate facial features of a person to detect a person'"'"'s face, said processor comparing the detected image of the person'"'"'s face with stored images, and causing a vehicle operation to be performed if a particular person'"'"'s face is identified;

wherein the infrared source is pulsed on for a predetermined period of time and a predetermined number of frames of video data and then is pulsed off for a predetermined period of time over the same predetermined number of frames of video data, and wherein the processor performs a frame differencing process where the frames video data when the infrared source is on are subtracted from the frames of video data when the infrared source is off so as to eliminate background.

2. A drowsy driver detection system for a vehicle, said system comprising:

an infrared source generating an infrared beam along a particular field-of-view in or around the vehicle, said source being pulsed on and off at predetermined intervals;

an infrared detector responsive to infrared radiation reflected from objects from the field-of-view of the infrared beam, said detector generating image signals from the reflected infrared radiation; and

a processor responsive to the image signals from the detector, said processor including face recognition software that images and identifies a plurality of separate facial features of a person to detect a person'"'"'s face, said processor comparing the detected image of the person'"'"'s face with stored images, said processor monitoring the eyes of the person and providing a signal if the person appears drowsy;

wherein the infrared source is pulsed on for a predetermined period of time and a predetermined number of frames of video data and then is pulsed off for a predetermined period of time over the same predetermined number of frames of video data, and wherein the processor performs a frame differencing process whore the frames video data when the infrared source is on are subtracted from the frames of video data when the infrared source is off so as to eliminate background.

3. An imaging system for a vehicle, said system comprising:

an infrared source generating an infrared beam, said infrared source being mounted at a forward location of the vehicle and providing a field-of-view in front of the vehicle;

an infrared detector responsive to infrared radiation reflected from objects in the field-of-view of the infrared beam, said detector generating image signals from the reflected infrared radiation; and

a processor responsive to the image signals from the detector, said processor including face recognition software that images and identifies a plurality of separate facial features of a person to detect a person'"'"'s face, said processor providing a signal if a person'"'"'s face is detected in front of the vehicle;

wherein the infrared source is pulsed on for a predetermined period of time and a predetermined number of frames of video data and then is pulsed off for a predetermined period of time over the same predetermined number of frames of video data, and wherein the processor performs a frame differencing process where the frames video data when the infrared source is on are subtracted from the frames of video data when the infrared source is off so as to eliminate background.

4. An imaging system for a vehicle, said system comprising;

an infrared source generating an infrared beam along a particular field-of-view in or around the vehicle;

an infrared detector responsive to infrared radiation reflected from objects in the field-of-view of the infrared beam, said detector generating image signals from the reflected infrared radiation; and

a processor responsive to the image signals from the detector, said processor including face recognition software that images and identifies a plurality of separate facial features of a person to detect a person'"'"'s face, said processor providing a signal if a person'"'"'s face is detected in a blind spot of the vehicle;

wherein the infrared source is pulsed on for a predetermined period of time and a predetermined number of frames of video data and then is pulsed off for a predetermined period of time over the same predetermined number of frames of video data, and wherein the processor performs a frame differencing process where the frames video data when the infrared source is on are subtracted from the frames of video data when the infrared source is off so as to eliminate background.

5. An imaging system for a vehicle, said system comprising:

an infrared source generating an infrared beam along a particular field-of-view in or around the vehicle;

an infrared detector responsive to infrared radiation reflected from objects in the field-of-view of the infrared beam, said detector generating image signals from the reflected infrared radiation; and

a processor responsive to the image signals from the detector, said processor including face recognition software that images and identifies a plurality of separate facial features of a person to detect a person'"'"'s face, said processor providing a recording of people within or around the vehicle during a crash event;

wherein the infrared source is pulsed on for a predetermined period of time and a predetermined number of frames of video data and then is pulsed off for a predetermined period of time over the same predetermined number of frames of video data, and wherein the processor performs a frame differencing process where the frames video data when the infrared source is on are subtracted from the frames of video data when the infrared source is off so as to eliminate background.

6. An imaging system for a vehicle, said system comprising:

an infrared source generating an infrared beam along a particular field-of-view in or around the vehicle;

an infrared detector responsive to infrared radiation reflected from objects in the field-of-view of the infrared beam, said detector generating image signals from the reflected infrared radiation; and

a processor responsive to the image signals from the detector, said processor including face recognition software that images and identifies a plurality of separate facial features of a person to detect a person'"'"'s face, said processor providing a signal if an incapacitated person is detected in the vehicle;

wherein the infrared source is pulsed on for a predetermined period of time and a predetermined number of frames of video data and then is pulsed off for a predetermined period of time over the same predetermined number of frames of video data, and wherein the processor performs a frame differencing process where the frames video data when the infrared source is on are subtracted from the frames of video data when the infrared source is off so as to eliminate background.

7. An imaging system for a vehicle, said system comprising,

an infrared source generating an infrared beam along a particular field-of-view in or around the vehicle;

an infrared detector responsive to infrared radiation reflected from objects in the field-of-view of the infrared beam, said detector generating image signals from the reflected infrared radiation; and

a processor responsive to the image signals from the detector, said processor including face recognition software that images and identifies a plurality of separate facial features of a person to detect a person'"'"'s face, said processor monitoring gestures of the detected person to cause a vehicle operation to be performed;

wherein the infrared source is pulsed on for a predetermined period of time and a predetermined number of frames of video data and then is pulsed off for a predetermined period of time over the same predetermined number of frames of video data, and wherein the processor performs a frame differencing process where the frames video data when the infrared source is on are subtracted from the frames of video data when the infrared source is off so as to eliminate background.