System and method for determining state of driver

• US 10,558,875 B2
• Filed: 11/06/2017
• Issued: 02/11/2020
• Est. Priority Date: 05/11/2017
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Motors Corporation (Kia Corp.)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Ryu, Seong Sook, Kim, Jin Kwon, Lee, Byoung Joon, Kim, Sam Yong
• Primary Examiner(s): Woldemariam, Aklilu K
• Application Number:
  US15/804,749
• Publication Number:
  US 20180330177A1
• Time in Patent Office: 827 Days
• Field of Search: 382100, 382154, 382209, 382255, 382201, 382118, 382115, 340575, 340576
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60Y 2400/90 Driver alarms
• CPC Class Codes: B62D 15/021 Determination of steering angle
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A system for determining a state of a driver includes a face tracking device configured to detect coordinates and a direction vector of a face at intervals of a sampling time from a face image of the driver, an event detecting device configured to detect valid behavior of a vehicle as an event, and a determination device configured to determine an impaired state of the driver based on a reflex response time of the driver when the event is detected by the event detecting device.

Claims

1. A system for determining a state of a driver, the system comprising:

a memory storing program instructions; and

a processor configured to execute the program instructions, the program instructions when executed configured to;

detect coordinates and a direction vector of a face at intervals of a sampling time from a face image of the driver,

detect a valid behavior of a vehicle as an event, and

determine an impaired state of the driver based on a reflex response time of the driver when the event is detected,

wherein the program instructions when executed are further configured to;

differentiate direction vectors of the face detected in a first time interval before the event occurs,

differentiate direction vectors of the face calculated in a second time interval after the event occurs,

determine the impaired state of the driver based on a variance value of differential values of a direction vector in the second time interval and a variance value of differential values of a direction vector in the first time interval, and

determine that the driver is in the impaired state when a sum of sampling times of differential values corresponding to a variance value which meets a threshold range of a reference variance value among variance values of the differential values of the direction vector in the second time interval is greater than a threshold value.

2. The system of claim 1, wherein the program instructions when executed are further configured to, before determining that the driver is in the impaired state:

set the variance value of the differential values of the direction vector in the first time interval to the reference variance value, and

sequentially calculate the variance values of the differential values of the direction vector in the second time interval.

3. The system of claim 1, further comprising an output device configured to warn the impaired state of the driver.

4. The system of claim 3, wherein the output device is configured to warning of the impaired state of the driver through at least one of a visual warning, an audible warning, or a tactile warning.

5. The system of claim 1, wherein the program instructions when executed are further configured to adjust a time when control of the vehicle is transferred to the driver when the system is mounted on an autonomous vehicle and the driver is in the impaired state.

6. The system of claim 1, wherein the program instructions when executed are further configured to store the coordinates and the direction vector of the face.

7. The system of claim 1, wherein the program instructions when executed are further configured to detect any one of a left turn of the vehicle, a right turn of the vehicle, driving on a curved road, or driving over a speed bump as the event.

8. The system of claim 7, wherein the program instructions when executed are further configured to detect any one of whether a steering angle is greater than a first reference value, whether a deviation between left and right wheel speeds is greater than a second reference value, or whether a lateral acceleration is greater than a third reference value as the event.

9. A method for determining a state of a driver, the method comprising:

detecting, by a face tracking device, coordinates and a direction vector of a face at intervals of a sampling time from a face image of the driver;

detecting, by an event detecting device, a valid behavior of a vehicle as an event; and

determining, by a determination device, an impaired state of the driver based on a reflex response time of the driver, when the event is detected,

wherein the determining comprises;

differentiating direction vectors of the face detected in a first time interval before the event occurs,

differentiating direction vectors of the face calculated in a second time interval after the event occurs,

setting a variance value of differential values of a direction vector in the first time interval to a reference variance value,

sequentially calculating variance values of differential values of a direction vector in the second time interval;

calculating a sum of sampling times of differential values corresponding to a variance value which meets a threshold range of the reference variance value among the calculated variance values, and

determining that the driver is in the impaired state when the calculated sum of the sampling times is greater than a threshold value.

10. The method of claim 9, further comprising warning, by an output device, of the impaired state of the driver through at least one of a visual warning, an audible warning, or a tactile warning.

11. The method of claim 9, further comprising adjusting, by the determination device, a time at which control of the vehicle is transferred to the driver when it is determined that the driver is in the impaired state.

12. The method of claim 9, further comprising storing the detected coordinates and direction vector of the face.

13. The method of claim 9, wherein the detecting of a valid behavior of a vehicle as an event comprises detecting any one of whether a steering angle is greater than a first reference value, whether a deviation between left and right wheel speeds is greater than a second reference value, or whether a lateral acceleration is greater than a third reference value as the event.