Driver state monitoring using corneal reflection detection

• US 10,346,697 B2
• Filed: 12/21/2016
• Issued: 07/09/2019
• Est. Priority Date: 12/21/2016
• Status: Active Grant

• Current Assignee: Hyundai America Technical Center, Inc. (Hyundai Mobis Co., Ltd.), Hyundai Motor Company (Hyundai Motor Group), Kia Motors Corporation (Kia Corp.)
• Original Assignee: Hyundai America Technical Center, Inc. (Hyundai Mobis Co., Ltd.)
• Inventors: Adibi, Sasan, Naserian, Mohammad
• Primary Examiner(s): Seth, Manav
• Application Number:
  US15/386,232
• Publication Number:
  US 20180173975A1
• Time in Patent Office: 930 Days
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/4809 Sleep detection, i.e. determining whether a subject is asleep or not
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/26 Incapacity
• CPC Class Codes: B60W 2720/10 Longitudinal speed
• CPC Class Codes: B60W 30/146 Speed limiting
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A method includes: sensing light information reflecting from an eye of a driver of a vehicle using an optical sensor mounted in the vehicle; measuring, by a control unit equipped in the vehicle, a plurality of eye parameters based on the sensed light information; calculating, by the control unit, an endangerment score indicating a driving state of the driver according to the plurality of eye parameters; determining, by the control unit, whether the calculated endangerment score exceeds an endangerment threshold; and executing, by the control unit, a corrective action in response to determining that the calculated endangerment score exceeds the endangerment threshold.

Claims

1. A method comprising:

sensing light information reflecting from an eye of a driver of a vehicle using an optical sensor mounted in the vehicle;

measuring, by a control unit equipped in the vehicle, a plurality of eye parameters based on the sensed light information;

calculating, by the control unit, an endangerment score indicating a driving state of the driver according to the plurality of eye parameters;

determining, by the control unit, whether the calculated endangerment score exceeds an endangerment threshold; and

executing, by the control unit, a corrective action in response to determining that the calculated endangerment score exceeds the endangerment threshold,

wherein the measuring of the plurality of eye parameters comprises measuring, by the control unit, a cornea stillness indicating a stillness of a cornea of the driver during a period of time based on the sensed light information.

2. The method of claim 1, wherein the optical sensor is mounted to a rear-view mirror of the vehicle.

3. The method of claim 2, wherein a light source is coupled to the optical sensor.

4. The method of claim 1, wherein the measuring of the plurality of eye parameters comprises:

measuring, by the control unit, a gaze alignment indicating an alignment of a gaze of the driver with a road in front of the vehicle based on the sensed light information.

5. The method of claim 4, wherein the measuring of the gaze alignment comprises:

determining, by the control unit, whether the gaze of the driver deviates from the road in front of the vehicle beyond a predefined acceptable amount.

6. The method of claim 1, wherein the measuring of the cornea stillness comprises:

determining, by the control unit, a number of movements of the cornea of the driver during the period of time.

7. The method of claim 1, wherein the measuring of the plurality of eye parameters comprises:

measuring, by the control unit, an eyelid closure indicating a closure amount of an eyelid of the driver during a period of time based on the sensed light information.

8. The method of claim 7, wherein the measuring of the eyelid closure comprises:

determining, by the control unit, a maximum amount of time the eyelid of the driver is closed during the period of time.

9. The method of claim 1, wherein the calculating of the endangerment score comprises:

measuring, by the control unit, a gaze alignment indicating an alignment of a gaze of the driver with a road in front of the vehicle based on the sensed light information;

measuring, by the control unit, a cornea stillness indicating a stillness of a cornea of the driver during a period of time based on the sensed light information;

measuring, by the control unit, an eyelid closure indicating a closure amount of an eyelid of the driver during a period of time based on the sensed light information; and

calculating, by the control unit, the endangerment score using the measured gaze alignment, cornea stillness, and eyelid closure.

10. The method of claim 1, further comprising:

controlling, by the control unit, the endangerment threshold according to a desired sensitivity.

11. The method of claim 1, further comprising:

adjusting, by the control unit, the endangerment threshold in real-time according to one or more factors selected from a group consisting of;

a driving direction, a driving speed, a time of day, a driving duration, a travel origin, a travel destination, a number of passengers in the vehicle, an amount of noise in the vehicle, and a mobile device usage.

12. The method of claim 1, further comprising:

determining, by the control unit, previously set endangerment thresholds; and

setting, by the control unit, the endangerment threshold based on the previously set endangerment thresholds.

13. The method of claim 1, further comprising:

applying, by the control unit, a filter to the sensed light information to remove noise therefrom.

14. The method of claim 1, wherein the measuring of the plurality of eye parameters comprises:

determining, by the control unit, a current speed of the vehicle; and

measuring, by the control unit, the plurality of eye parameters based further on the determined current speed of the vehicle.

15. The method of claim 1, wherein the executing of the corrective action comprises:

providing a visual, audible, or tangible warning indicator to the driver.

16. The method of claim 1, wherein the executing of the corrective action comprises:

reducing, by the control unit, a speed of the vehicle.

17. The method of claim 15, wherein the executing of the corrective action comprises:

determining, by the control unit, whether the calculated endangerment score continues to exceed the endangerment threshold after reducing the speed of the vehicle; and

further reducing, by the control unit, the speed of the vehicle when the calculated endangerment score continues to exceed the endangerment threshold.

18. A system comprising:

an optical sensor mounted in a vehicle configured to sense light information reflecting from an eye of a driver of the vehicle; and

a control unit equipped in the vehicle and configured to;

measure a plurality of eye parameters based on the sensed light information;

measure a cornea stillness indicating a stillness of a cornea of the driver during a period of time based on the sensed light information, the plurality of eye parameters including the cornea stillness;

calculate an endangerment score indicating a driving state of the driver according to the plurality of eye parameters;

determine whether the calculated endangerment score exceeds an endangerment threshold; and

execute a corrective action in response to determining that the calculated endangerment score exceeds the endangerment threshold.

19. A non-transitory computer readable medium containing program instructions executable by a control unit equipped in a vehicle, wherein the program instructions when executed cause the control unit to:

measure a plurality of eye parameters based on light information reflecting from an eye of a driver of the vehicle sensed by an optical sensor mounted in the vehicle;

measure a cornea stillness indicating a stillness of a cornea of the driver during a period of time based on the sensed light information, the plurality of eye parameters including the cornea stillness;

calculate an endangerment score indicating a driving state of the driver according to the plurality of eye parameters;

determine whether the calculated endangerment score exceeds an endangerment threshold; and

execute a corrective action in response to determining that the calculated endangerment score exceeds the endangerment threshold.