Human presence detection, identification and tracking using a facial feature image sensing system for airbag deployment

• US 6,904,347 B1
• Filed: 06/29/2000
• Issued: 06/07/2005
• Est. Priority Date: 06/29/2000
• Status: Expired due to Fees

• Current Assignee: TRW Limited (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Original Assignee: TRW Limited (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Inventors: Shreve, Gregory A., Niesen, Joseph W., Dunbridge, Barry, McIver, George W., Berenz, John J.
• Primary Examiner(s): Cuchlinski, Jr., William A.
• Assistant Examiner(s): MANCHO, RONNIE M
• Application Number:
  US09/607,090
• Time in Patent Office: 1,804 Days
• Field of Search: 701/45, 701/47, 701/46, 340/870.37, 340/825.36, 340/436, 342/71, 342/84, 342/104, 342/107, 280/735, 280/736, 280/739, 250/559.22, 250/559.23, 250/559.27, 324/65.B, 324/663, 324/61, 180/272, 180/273, 180/271, 382/100, 382/103, 382/291, 382/104, 382/111, 382/115, 382/118, 382/200
• US Class Current: 701/45
• CPC Class Codes: B60R 21/01534 using electromagneticwaves, e.g. infrared
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01552 detecting position of specific human body parts, e.g. face, eyes or hands

Abstract

A vehicle occupant airbag deployment system (50) that detects, identifies and tracks a person (16) in the passenger seat (18) of a vehicle (12), and provides a signal for no fire, soft fire or hard fire of the airbag (20) depending on the location of the person (16) in a crash event. The airbag deployment system (50) employs infrared emitters (30) that emit an infrared signal towards the passenger seat (18) of the vehicle (12) and an infrared detector (34) that receive reflected radiation from objects in the seat (18). Processing circuitry (52), including face recognition software, is employed to detect human face features to provide the necessary detection and tracking of the person (16). In this manner, the system (50) prevents the airbag (20) from firing if the seat (18) is not occupied by a person (16), prevents the airbag (20) from firing if a person (16) is detected, but is too close to the airbag (20), and provides a soft fire if a person (16) is detected, but is within a soft fire range of the airbag (20).

Claims

1. A system for deploying a passenger side airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including face recognition software that identifies, acquires and tracks specific and individual facial features of the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel;

wherein the infrared source is a cluster of infrared light emitting diodes;

and wherein the infrared detector is an electronic video camera.

2. The system according to claim 1 wherein the cluster of LEDs and the electronic camera are provided as a single unit mounted to the instrument panel in front of the passenger seat.

3. A system for deploying a passenger side airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including face recognition software that identifies, acquires and tracks specific and individual facial features of the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel;

wherein the processor tracks the passengers eyes;

and wherein the processor determines the distance the passenger is from the instrument panel by the separation distance of the passengers eyes.

4. A system for deploying a passenger side airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including face recognition software that identifies, acquires and tracks specific and individual facial features of the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel;

and wherein the processor determines the distance the passenger is from the instrument panel by the distance between two locations on the passenger'"'"'s face.

5. A system for deploying a passenger side airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including face recognition software that identifies, acquires and tracks specific and individual facial features of the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel;

wherein the infrared detector includes a pixel array of photodiodes.

6. A system for deploying a passenger side airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including face recognition software that identifies, acquires and tracks specific and individual facial features of the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel;

wherein the infrared source is pulsed on for a predetermined time at predetermined intervals;

and wherein the pulsed source and the detector are synchronized such that a difference of successive frames represents an image free of ambient light variations.

7. The system according to claim 6 where said difference of successive frames is directly performed by the detector.

8. The system according to claim 6 wherein frames of data are stored in the detector when the source is on and the source is off to subtract out visible light.

9. A system for deploying a passenger side airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation, said source and detector being part of a single unit mounted to the instrument panel; and

a processor responsive to the image from the detector, said processor including face recognition software that identifies, acquires and tracks facial features of the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel, wherein the processor tracks the passenger'"'"'s eyes and determines the distance the passenger is from the instrument panel by the separation distance of the passenger'"'"'s eyes.

10. The system according to claim 9 wherein the infrared source is a cluster of infrared light emitting diodes.

11. The system according to claim 9 wherein the detector is an electronic camera.

12. The system according to claim 9 further comprising an infrared filter positioned in front of the detector.

13. The system according to claim 9 wherein the infrared source is pulsed on for a predetermined time at predetermined intervals.

14. The system according to claim 13 wherein the pulsed source and the detector are synchronized such that a difference of successive frames represents an image free of ambient light variations.

15. The system according to claim 14 wherein said difference of successive frames is directly performed by the detector.

16. The system according to claim 9 wherein frames of data are stored when the source is on and the source is off to subtract out visible light.

17. The system according to claim 9 wherein the detector includes a pixel array of photodiodes.

18. A method of determining whether to deploy a passenger side airbag in a vehicle, said method comprising the steps of:

directing a beam of infrared radiation towards a passenger seat of the vehicle;

detecting reflected infrared radiation from objects in the passenger seat;

using face recognition processing to determine whether a person is in the passenger seat, said face recognition processing identifying specific and individual facial features of the passenger; and

determining whether to fire the airbag by whether a face is detected and the range of the person relative to the airbag;

wherein the step of directing the beam of radiation includes pulsing the beam of radiation;

and further comprising the step of employing a frame differencing technique to subtract ambient radiation from detected images.

19. A system for deploying a passenger airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

a cluster of infrared light emitting diodes generating an infrared beam directed towards the passenger seat;

an electronic video camera responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including human presence detection software that identifies, acquires and tracks the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel.

20. The system according to claim 19 wherein the cluster of LEDs and the electronic camera are provided as a single unit mounted to the instrument panel in front of the passenger seat.

21. The system according to claim 19 wherein the processor tracks the passenger'"'"'s eyes.

22. The system according to claim 19 further comprising an infrared filter positioned in the camera to filter the received radiation.

23. The system according to claim 19 wherein the light emitting diodes are pulsed on for a predetermined time at predetermined intervals.

24. The system according to claim 19 wherein the processor includes a database that stores information about specific individuals, said processor using the database to identify the passenger.

25. A system for deploying a passenger airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including human presence detection software that identifies, acquires and tracks the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel, wherein the processor tracks the passenger'"'"'s eyes and determines the distance the passenger is from the instrument panel by the separation distance of the passenger'"'"'s eyes.

26. A system for deploying a passenger airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including human presence detection software that identifies, acquires and tracks the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel, wherein the processor determines the distance the passenger is from the instrument panel by the distance between two locations on the passengers face.

27. The system according to claim 26 wherein the processor tracks the passenger'"'"'s eyes.

28. The system according to claim 26 further comprising an infrared filter positioned in front of the detector.

29. The system according to claim 26 wherein the infrared source is pulsed on for a predetermined time at predetermined intervals.

30. The system according to claim 26 wherein the processor includes a database that stores information about specific individuals, said processor using the database to identify the passenger.

31. A system for deploying a passenger airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation, wherein the infrared detector includes a pixel array of photodiodes; and

a processor responsive to the image from the detector, said processor including human presence detection software that identifies, acquires and tracks the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel.

32. The system according to claim 31 wherein the processor tracks the passenger'"'"'s eyes.

33. The system according to claim 31 further comprising an infrared filter positioned in front of the detector.

34. The system according to claim 31 wherein the infrared source is pulsed on for a predetermined time at predetermined intervals.

35. The system according to claim 31 wherein the processor includes a database that stores information about specific individuals, said processor using the database to identify the passenger.

36. A system for deploying a passenger airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat, wherein the infrared source is pulsed on for a predetermined time at predetermined intervals;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, wherein the infrared source is pulsed on for a predetermined time at predetermined intervals, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation, wherein the pulsed source and the detector are synchronized such that a difference of successive frames represents an image free of ambient light variations; and

a processor responsive to the image from the detector, said processor including human presence detection software that identifies, acquires and tracks the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel.

37. The system according to claim 36 wherein said difference of successive frames is directly performed by the detector.

38. The system according to claim 36 wherein the processor tracks the passenger'"'"'s eyes.

39. The system according to claim 36 further comprising an infrared filter positioned in front of the detector.

40. The system according to claim 36 wherein the infrared source is pulsed on for a predetermined time at predetermined intervals.

41. The system according to claim 36 wherein the processor includes a database that stores information about specific individuals, said processor using the database to identify the passenger.

42. A system for deploying a passenger airbag in a vehicle for restraining a passenger, said airbag being mounted in an instrument panel of the vehicle in front of a passenger seat, said system comprising:

an infrared source generating an infrared beam directed towards the passenger seat;

an infrared detector responsive to infrared radiation reflected from objects in the passenger seat, said detector generating signals indicative of an image of the objects in the passenger seat based on the reflected infrared radiation; and

a processor responsive to the image from the detector, said processor including human presence detection software that identifies, acquires and tracks the passenger, said processor causing the airbag to deploy in a crash event if the passenger is detected in the passenger seat and is a predetermined distance away from the instrument panel, wherein frames of data are stored in the detector when the source is on and the source is off to subtract out visible light.

43. The system according to claim 42 wherein the processor tracks the passenger'"'"'s eyes.

44. The system according to claim 42 further comprising an infrared filter positioned in front of the detector.

45. The system according to claim 42 wherein the infrared source is pulsed on for a predetermined time at predetermined intervals.

46. The system according to claim 42 wherein the processor includes a database that stores information about specific individuals, said processor using the database to identify the passenger.

47. A method of determining whether to deploy a passenger airbag in a vehicle, said method comprising the steps of:

directing a pulsed beam of infrared radiation towards a passenger seat of the vehicle;

detecting reflected infrared radiation from objects in the passenger seat;

employing a frame differencing technique to subtract ambient radiation from detected images;

using face recognition processing to determine whether a person is in the passenger seat; and

determining whether to fire the airbag by whether a face is detected and the range of the person relative to the airbag.

48. The method according to claim 47 wherein the step of using face recognition processing includes recognizing and determining a person'"'"'s face by his or her eyes.

49. The method according to claim 47 wherein the step of determining whether to fire the airbag includes determining whether to soft fire or hard fire the airbag by the distance of the person from the detector and/or the size of the person.