Operator identity, intoxication and drowsiness monitoring system and method

• US 5,729,619 A
• Filed: 08/08/1995
• Issued: 03/17/1998
• Est. Priority Date: 08/08/1995
• Status: Expired due to Term

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Corporation
• Inventors: Puma, Samuel C.
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Tadayon, Bijan
• Application Number:
  US08/512,593
• Time in Patent Office: 952 Days
• Field of Search: 382/115, 382/116, 382/117, 382/118, 382/119, 382/120, 382/124, 382/126, 382/127, 382/321, 382/181, 382/190, 382/192, 382/209, 382/217, 382/224, 382/278, 382/312, 382/317, 382/318, 382/319, 382/323, 128/665, 128/733, 128/719, 180/272, 180/268, 340/576, 340/575, 340/573, 364/569, 351/205, 351/206, 351/210, 436/132
• US Class Current: 382/115
• CPC Class Codes: B60K 28/063 preventing starting of vehicles
• CPC Class Codes: B60R 25/04 operating on the propulsion system, e.g. engine or drive motor
• CPC Class Codes: B60R 25/25 using biometry
• CPC Class Codes: B60R 25/252 Fingerprint recognition
• CPC Class Codes: B60R 25/255 Eye recognition
• CPC Class Codes: B60R 25/305 using a camera
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

An integrated vehicle operator monitoring system for a vehicle including a device for monitoring operator identity to ensure a vehicle operator is authorized to operate the vehicle, a device for monitoring operator intoxication, and a device for monitoring operator drowsiness. There is also a device for preventing the operator from operating the vehicle when the operator identity monitoring device indicates the operator is not authorized to operate the vehicle and/or the operator intoxication monitoring device indicates the operator is intoxicated. In addition, an alarm capable of being activated when the operator drowsiness monitoring device indicates the vehicle operator is drowsy, is included.

Claims

1. An integrated vehicle operator monitoring system for a vehicle, comprising:

an optical operator intoxication monitor optically capable of monitoring reflected and emitted light patterns from an eye of the vehicle operator and comparing the monitored light patterns to reflected and emitted light patterns indicative of an intoxicated operator so as to determine if the operator is too intoxicated to operate the vehicle.

2. The monitoring system of claim 1, further comprising:

an optical operator identity monitor capable of determining if a vehicle operator is authorized to operate the vehicle.

3. The monitoring system of claim 1, further comprising:

an optical operator drowsiness monitor capable of monitoring the vehicle operator for drowsiness while the vehicle operator is operating the vehicle.

4. The monitoring system of claim 2, wherein the operator identity monitor comprises:

an optical sensor having a sensing surface;

operator identity monitoring optics capable of focusing an image of an eye of the vehicle operator on the sensing surface of the optical sensor;

a memory for storing data associated with predetermined characteristics of the irises of vehicle operators'"'"' eyes who are authorized to operate the vehicle; and

,

an authorized operator determination apparatus capable of determining if the vehicle operator is authorized to drive the vehicle from a signal output by the optical sensor representing an image of the vehicle operator'"'"'s eye and the stored data.

5. The monitoring system of claim 4, wherein the authorized operator determination apparatus comprises:

a processor comprising,

a first processor portion for deriving data corresponding to said predetermined characteristics of the vehicle operator'"'"'s iris from the signal output by the optical sensor representing an image of the vehicle operator'"'"'s eye,

a second processor portion for comparing the derived data with the stored data associated with the irises of authorized vehicle operators, and,

a third processor portion for indicating either that (i) the vehicle operator is authorized to operate the vehicle whenever a substantial match is found between the derived data and the stored data, or (ii) the vehicle operator is not authorized to operate the vehicle when no substantial match is found between the derived data and the stored data.

6. The monitoring system of claim 4, wherein the operator identity monitor further comprises:

an illuminator for illuminating the eye of the vehicle operator to highlight and contrast said predetermined characteristics of the vehicle operator'"'"'s iris.

7. The monitoring system of claim 6, wherein:

the illuminator is capable of producing at least one wavelength of light in a range from about near infrared to near ultraviolet.

8. The monitoring system of claim 4, further comprising:

a housing; and

an indicator capable of indicating to the vehicle operator a position in reference to the housing of the integrated vehicle operator monitoring system where the operator is required to position at least one of the operator'"'"'s eyes such that the operator identity monitoring optics provides a best possible focused image at the sensing surface to the optical sensor.

9. The monitoring system of claim 8, wherein the indicator comprises:

an image projector capable of projecting a two-part image wherein the two parts of the image come to a focus only at said position where the operator is required to position said one at least of the operator'"'"'s eyes.

10. The monitoring system of claim 1, wherein the operator intoxication monitor comprises:

an optical sensor having a sensing surface;

operator intoxication monitoring optics capable of collecting light reflected and emitted from an eye of the vehicle operator, spectrally separating the collected light to form an absorbance and fluorescence emission pattern, and focusing the spectrally separated light on the sensing surface of the optical sensor;

a memory for storing data sets respectively associated with an absorbance and fluorescence emission pattern formed from spectrally separated light reflected and emitted from an eye of an intoxicated person, wherein each of said stored data sets represents one of (i) a particular intoxicant and an associated concentration of that intoxicant in the body of the vehicle operator, or (ii) a particular combination of intoxicants and associated concentrations of that intoxicant in the body of the vehicle operator; and

,

a comparator for determining if the vehicle operator is too intoxicated to drive the vehicle from a signal output by the optical sensor and the stored data sets.

11. The monitoring system of claim 10, wherein the comparator comprises:

a processor comprising,

a first processor portion for deriving data associated with an absorbance and fluorescence emission pattern formed from spectrally separated light reflected and emitted from the eye of the vehicle operator from the signal output by the optical sensor,

a second processor portion for comparing the derived data with the stored data sets, and,

a third processor portion for indicating either that (i) the vehicle operator is too intoxicated to operate the vehicle whenever a substantial match is found between the derived data and any stored data set, or (ii) the vehicle operator is not too intoxicated to operate the vehicle when no substantial match is found between the derived data and any stored data set.

12. The monitoring system of claim 10, wherein:

the operator intoxication monitoring optics is capable of focusing the spectrally separated light on the sensing surface of the optical sensor with a resolution of at least 5.0 nm.

13. The monitoring system of claim 10, wherein the operator intoxication monitor further comprises:

an illuminator for illuminating the eye of the vehicle operator with eye-safe levels of light having a spectral range from about near infrared to near ultraviolet.

14. The monitoring system of claim 10, further comprising:

a housing; and

an indicator capable of indicating to the vehicle operator a position in reference to the housing of the integrated vehicle operator monitoring system where the operator is required to position one of the operator'"'"'s eyes such that the operator intoxication monitoring optics provides a best possible focused image at the sensing surface to the optical sensor.

15. The monitoring system of claim 14, wherein the indicator comprises:

an image projector capable of for projecting a two-part image wherein the two parts of the image come to a focus only at said position where the operator is required to position the one of the operator'"'"'s eyes.

16. The monitoring system of claim 3, wherein the operator drowsiness monitor comprises:

an optical sensor having a sensing surface;

operator drowsiness monitoring optics capable of focusing an image of the vehicle operator'"'"'s head and eyes on the sensing surface of the optical sensor;

a memory for storing image data associated with physical characteristics of a vehicle operator'"'"'s head and eyes which are one of (i) indicative of drowsiness, or (ii) indicative of alertness; and

,

an isolator capable of isolating from a signal output by the optical sensor that portion corresponding to said physical characteristics;

an indicator for indicating from the isolated portion of the signal and the stored image data which of the physical characteristics exhibited by the vehicle operator are indicative of drowsiness.

17. The monitoring system of claim 16, wherein the indicator comprises:

a processor comprising,

a first processor portion for deriving image data corresponding to said physical characteristics of the vehicle operator'"'"'s head and eyes from the signal output by the optical sensor; and

a second processor portion for comparing the derived image data with the stored image data.

18. The monitoring system of claim 16, further comprising:

a performance monitor for monitoring performance characteristics associated with the vehicle operator'"'"'s actions to control of the vehicle; and

a discriminator for ascertaining which of the performance characteristics exhibited by the vehicle operator are indicative of drowsiness.

19. The monitoring system of claim 18, further comprising:

a physical and performance characteristic monitor for determining from the physical and performance characteristics exhibited by the vehicle operator which are indicative of drowsiness if the vehicle operator is in danger of falling asleep and providing an output signal indicating operator drowsiness.

20. The monitoring system of claim 19, wherein:

at least one of the physical and performance characteristics exhibited by the vehicle operator must be indicative of drowsiness to cause the physical and performance characteristic monitor to output said output signal indicating operator drowsiness.

21. The monitoring system of claim 19, wherein:

particular ones of the physical and performance characteristics exhibited by the vehicle operator provide definite indications of drowsiness to a high degree of confidence, while the remainder provide a less definite indication of drowsiness; and

the physical and performance characteristic monitor outputs said output signal indicating operator drowsiness whenever one of (i) any one of the characteristics providing a definite indication of drowsiness is present, or (ii) two or more of the characteristics providing a less definite indication of drowsiness are present.

43. The monitoring system of claim 3, further comprising:

an alarm capable of being activated whenever the operator drowsiness monitoring means indicates the vehicle operator is drowsy.

44. The monitoring system of claim 1, further comprising:

a vehicle disabling apparatus for preventing the operator from operating the vehicle whenever the operator intoxication monitor indicates the vehicle operator is too intoxicated to operate the vehicle.

45. The monitoring system of claim 2, further comprising:

a vehicle disabling apparatus for preventing the operator for operating the vehicle whenever the operator identity monitor indicates the vehicle operator is not authorized to operate the vehicle.

22. An integrated vehicle monitoring system for a vehicle comprising:

an optical operator drowsiness monitor capable of monitoring the vehicle operator for drowsiness while the vehicle operator is operating the vehicle;

an optical operator identity monitor capable of determining if a vehicle operator is authorized to operate the vehicle; and

an optical intoxication monitor capable of monitoring reflected and emitted light patterns from an eye of the vehicle operator and comparing the monitored light patterns to reflected and emitted light patterns indicative of an intoxicated operator so as to determine if the operator is too intoxicated to operate the vehicle.

46. The monitoring system of claim 22, wherein:

the operator identity monitor comprises;

an operator identification optical sensor having a sensing surface,

operator identity monitoring optics capable of focusing an image of an eye of the vehicle operator on the sensing surface of the optical sensor,

a memory for storing data associated with predetermined characteristics of the irises of vehicle operators'"'"' eyes who are authorized to operate the vehicle, and,

an authorized operator determination apparatus capable of determining if the vehicle operator is authorized to drive the vehicle from a signal output by the optical sensor representing an image of the vehicle operator'"'"'s eye and the stored data;

the operator intoxication monitor comprises;

an operator intoxication optical sensor having a sensing surface,

operator intoxication monitor capable of collecting light reflected and emitted from an eye of the vehicle operator, spectrally separating the collected light to form an absorbance and fluorescence emission pattern, and focusing the spectrally separated light on the sensing surface of the optical sensor,

a memory for storing data sets respectively associated with an absorbance and fluorescence emission pattern formed from spectrally separated light reflected and emitted from an eye of an intoxicated person, wherein each of said stored data sets represents one of (i) a particular intoxicant and an associated concentration of that intoxicant in the body of the vehicle operator, or (ii) a particular combination of intoxicants and associated concentrations of that intoxicant in the body of the vehicle operator, and,

a comparator for determining if the vehicle operator is too intoxicated to drive the vehicle from a signal output by the optical sensor and the stored data sets; and

the operator drowsiness monitor comprises;

an operator drowsiness optical sensor having a sensing surface;

operator drowsiness monitoring optics capable of focusing an image of the vehicle operator'"'"'s head and ayes on the sensing surface of the optical sensor,

a memory for storing image data associated with physical characteristics of a vehicle operator'"'"'s head and eyes which are one of (i) indicative of drowsiness, or (ii) indicative of alertness,

an isolator capable of isolating from a signal output by the optical sensor that portion corresponding to said physical characteristics, and

an indicator for indicating from the isolated portion of the signal and the stored image data which of the physical characteristics exhibited by the vehicle operator are indicative of drowsiness.

47. The monitoring system of claim 22, further comprising:

an alarm capable of being activated whenever the operator drowsiness monitor indicates the vehicle operator is drowsy.

50. The monitoring system of claim 46, wherein only one of the operator identity monitoring optics, operator intoxication monitoring optics, and operator drowsiness monitoring optics is in registration with the optical sensor at a time, the system further comprising:

means for moving the identity monitoring optics, operator intoxication monitoring optics, and operator drowsiness monitoring optics into registration with the optical sensor during operation of the means for monitoring operator identity, means for monitoring operator intoxication, and means for monitoring operator drowsiness, respectively.

23. A method of monitoring an operator of a vehicle, comprising the step of:

monitoring operator intoxication by monitoring reflected and emitted light patterns from an eye of the vehicle operator and comparing the monitored light patterns to reflected and emitted light patterns indicative of an intoxicated operator so as to determine if the operator is too intoxicated to operate the vehicle.

24. The method of claim 23, further comprising the step of:

preventing the operator from operating the vehicle whenever the vehicle operator is intoxicated.

25. The method of claim 23, wherein the monitoring method includes a step of monitoring operator drowsiness, and wherein the step of monitoring operator drowsiness comprises the steps of:

focusing an image of the vehicle operator'"'"'s head and eyes on a sensing surface of an optical sensor;

storing image data associated with physical characteristics of a vehicle operator'"'"'s head and eyes which are one of (i) indicative of drowsiness, or (ii) indicative of alertness; and

,

isolating from a signal output by the optical sensor that portion corresponding to said physical characteristics;

indicating from the isolated portion of the signal and the stored image data that of the physical characteristics exhibited by the vehicle operator are indicative of drowsiness.

26. The method of claim 23, wherein the monitoring method includes a step of monitoring operator identity, and wherein the step of monitoring operator identity comprises the steps of:

focusing an image of an eye of the vehicle operator on a sensing surface of the optical sensor;

storing data associated with predetermined characteristics of the irises of vehicle operators'"'"' eyes who are authorized to operate the vehicle; and

,

determining if the vehicle operator is authorized to drive the vehicle from a signal output by the optical sensor representing an image of the vehicle operator'"'"'s eye and the stored data.

27. The method of claim 26, wherein the determining step comprises the step of:

deriving data corresponding to said predetermined characteristics of the vehicle operator'"'"'s iris from the signal output by the optical sensor representing an image of the vehicle operator'"'"'s eye;

comparing the derived data with the stored data associated with the irises of authorized vehicle operators; and

,

indicating either that (i) the vehicle operator is authorized to operate the vehicle whenever a substantial match is found between the derived data and the stored data, or (ii) the vehicle operator is not authorized to operate the vehicle when no substantial match is found between the derived data and the stored data.

28. The method of claim 26, wherein the step of monitoring operator identity further comprises the step of:

illuminating the eye of the vehicle operator to highlight and contrast said predetermined characteristics of the vehicle operator'"'"'s iris.

29. The method of claim 28, wherein the illuminating step comprises the step of:

illuminating the eye of the vehicle operator with a light having at least one wavelength in a range from about near infrared to near ultraviolet.

32. The method of claim 27, wherein:

the focusing step comprises first focusing an image of a first eye of the vehicle operator on the sensing surface of the optical sensor, then focusing an image of the second eye of the vehicle operator on the sensing surface of the optical sensor; and

,

the indicating step comprises indicating either that (i) the vehicle operator is authorized to operate the vehicle whenever a substantial match is found between the derived data and the stored data for at least one of said first and second eyes of the vehicle operator, or (ii) the vehicle operator is not authorized to operate the vehicle when no substantial match is found between the derived data and the stored data for either of the vehicle operator'"'"'s eyes.

33. The method of claim 23, wherein the step of monitoring operator intoxication comprises the steps of:

collecting light reflected and emitted from an eye of the vehicle operator;

spectrally separating the collected light to form an absorbance and fluorescence emission pattern;

focusing the spectrally separated light on a sensing surface of an optical sensor;

sensing the respective intensities of various groups of the spectral components of the spectrally separated light;

producing a signal representative of said respective intensities;

storing data sets respectively associated with an absorbance and fluorescence emission pattern formed from spectrally separated light reflected and emitted from an eye of an intoxicated person, wherein each of said stored data sets represents one of (i) a particular intoxicant and an associated concentration of that intoxicant in the body of the vehicle operator, or (ii) a particular combination of intoxicants and associated concentrations of that intoxicant in the body of the vehicle operator; and

,

determining if the vehicle operator is too intoxicated to drive the vehicle from a signal output by the optical sensor and the stored data sets.

34. The method of claim 33, wherein the determining step comprises the steps of:

deriving data associated with an absorbance and fluorescence emission pattern formed from spectrally separated light reflected and emitted from the eye of the vehicle operator from the signal representative of said respective intensities of said various groups of the spectral components of the spectrally separated light;

comparing the derived data with the stored data sets, and,

indicating either that (i) the vehicle operator is too intoxicated to operate the vehicle whenever a substantial match is found between the derived data and any stored data set, or (ii) the vehicle operator is not too intoxicated to operate the vehicle when no substantial match is found between the derived data and any stored data set.

35. The method of claim 33, further comprising the step of:

illuminating the eye of the vehicle operator with eye-safe levels of light having a spectral range from about near infrared to near ultraviolet.

36. The method of claim 33, further comprising the step of:

indicating to the vehicle operator a position in reference to the housing of the integrated vehicle operator monitoring system where the operator is required to position one of the operator'"'"'s eyes such that the operator intoxication monitoring optics provides a best possible focused image at the sensing surface to the optical sensor.

37. The method of claim 25, further comprising the step of:

activating an alarm whenever the operator is drowsy.

38. The method of claim 25, wherein the indicating step comprises:

deriving image data corresponding to said physical characteristics of the vehicle operator'"'"'s head and eyes from the signal output by the optical sensor,

comparing the derived image data with the stored image data.

39. The method of claim 25, further comprising the steps of:

monitoring performance characteristics associated with the vehicle operator'"'"'s actions to control of the vehicle; and

ascertaining which of the performance characteristics exhibited by the vehicle operator are indicative of drowsiness.

40. The method of claim 39, further comprising the step of:

determining from the physical and performance characteristics exhibited by the vehicle operator which are indicative of drowsiness if the vehicle operator is in danger of falling asleep; and

providing an output signal indicating operator drowsiness.

41. The method of claim 40, wherein the determining step comprises:

providing said output signal indicating operator drowsiness whenever at least one of the physical and performance characteristics exhibited the vehicle operator is indicative of drowsiness.

42. The method of claim 40, wherein:

particular ones of the physical and performance characteristics exhibited by the vehicle operator provide definite indications of drowsiness to a high degree of confidence, while the remainder provide a less definite indication of drowsiness; and

wherein

the determining step comprises providing said output signal indicating operator drowsiness whenever one of (i) any one of the characteristics providing a definite indication of drowsiness is present, or (ii) two or more of the characteristics providing a less definite indication of drowsiness are present.

48. The method of claim 26, further comprising the step of:

preventing the operator from operating the vehicle whenever the vehicle operator is not authorized to operate the vehicle.

30. The method of 26, further comprising the step of:

indicating to the vehicle operator a position in reference to the housing of the integrated vehicle operator monitoring system where the operator is required to position at least one of the operator'"'"'s eyes such that the operator identity monitoring optics provides a best possible focused image at the sensing surface to the optical sensor.

31. The method of claim 30, wherein the indicating step comprises the step of:

projecting a two-part image wherein the two parts of the image come to a focus only at said position where the operator is required to position the at least one of the operator'"'"'s eyes.

49. A method of monitoring an operator of a vehicle comprising the steps of:

monitoring operator identity to determine if a vehicle operator is authorized to operate the vehicle;

monitoring operator drowsiness wherein said operator drowsiness monitoring is performed while the vehicle operator is operating the vehicle; and

monitoring operator intoxication by monitoring reflected and emitted light patterns from an eye of the vehicle operator and comparing the monitored light patterns to reflected and emitted light patterns indicative of an intoxicated operator so as to determine if the operator is too intoxicated to operate the vehicle.