System and method for driver distraction determination

• US 10,246,014 B2
• Filed: 11/07/2017
• Issued: 04/02/2019
• Est. Priority Date: 11/07/2016
• Status: Active Grant

• Current Assignee: Nauto, Inc.
• Original Assignee: Nauto, Inc.
• Inventors: Levkova, Ludmila, Heck, Stefan, Alpert, Benjamin O., Satzoda, Ravi Kumar, Sathyanarayana, Suchitra, Sekar, Vivek
• Primary Examiner(s): Sherwin, Ryan W
• Application Number:
  US15/805,348
• Publication Number:
  US 20180126901A1
• Time in Patent Office: 511 Days
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

A method for determining distraction of a driver of a vehicle, including sampling sensor measurements at an onboard system of the vehicle; generating an output indicative of a distracted state; determining that the driver of the vehicle is characterized by the distracted state, based on the output; generating, at a second distraction detection module of a remote computing system, a second output indicative that the driver is characterized by the distracted state, based on the sensor measurements; computing a distraction score, at a scoring module of the remote computing system, in response to generating the second output and based on the sensor measurements and the distracted state.

Claims

1. A method for determining distraction of a driver of a vehicle, comprising:

sampling sensor measurements at an onboard system of the vehicle, wherein the sensor measurements comprise an image of the driver;

receiving the image at a face detection module of the onboard system, and generating a first output, wherein the first output is binary between one of a presence and an absence of a face in the image;

receiving the image at a driver distraction module of the onboard system, and generating a second output indicative of a distracted state depicted in the image based on identifying a gaze direction of the driver, and wherein the second output is generated in parallel with and independently of the first output;

determining that the driver of the vehicle is characterized by the distracted state, based on a combination of the first output, wherein the first output indicates the presence of the face in the image, and the second output;

determining a duration of the gaze direction;

determining a score of the distracted state based on the duration of the gaze direction exceeding a threshold duration; and

notifying the driver, at an output of the onboard system of the vehicle and in substantially real-time, based on the distracted state and the score of the distracted state.

2. The method of claim 1, wherein the threshold duration is predetermined by a remote entity associated with the driver, and further comprising transmitting a notification to the remote entity based on the determined score of the distracted state.

3. The method of claim 1, wherein the sensor measurements comprise an accelerometer output, and further comprising:

determining driver behavior at a driver behavior analysis module of the onboard system based on the accelerometer output; and

wherein determining that the driver of the vehicle is characterized by the distracted state is based on the determined driver behavior.

4. The method of claim 1, further comprising transmitting the sensor measurements to a remote computing system in response to the score of the distracted state exceeding a threshold score.

5. The method of claim 4, wherein transmitting the sensor measurements to the remote computing system is performed substantially instantaneously subsequent to determining that the score of the distracted state exceeds the threshold score.

6. The method of claim 4, further comprising processing the sensor measurements at a second driver distraction module of the remote computing system to generate a third output indicative that the driver is characterized by the distracted state.

7. The method of claim 6, further comprising transmitting a notification to a remote entity associated with the driver, in response to generating the third output, wherein the notification comprises the distracted state, the score of the distracted state, the context of the distracted state, and a time point of distracted state determination.

8. The method of claim 4, further comprising computing a driver score, at a scoring module of the remote computing system, based on the sensor measurements, the determined score, and a time history of scores associated with the driver.

9. A method for determining distraction of a driver of a vehicle, comprising:

sampling sensor measurements at an onboard system of the vehicle, during a driving session, wherein the sensor measurements comprise a sequence of images including an image of the driver;

receiving the image at a face detection module of the onboard system, and generating a first output, wherein the first output is binary between one of a presence and an absence of a face in the image;

receiving the sequence of images at a driver distraction module of the onboard system, and generating a second output indicative of a distracted state depicted in the image based on computing a rate of change of a gaze direction of the driver from the sequence of images, wherein the second output is generated in parallel with and independently of the first output;

determining that the driver of the vehicle is characterized by the distracted state, based on a combination of the first output, wherein the first output indicates the presence of the face in the image, and the second output;

transmitting the sensor measurements to a remote computing system in response to determining that the driver of the vehicle is characterized by the distracted state;

processing the sensor measurements at a second driver distraction module of the remote computing system to generate a third output indicative that the driver is characterized by the distracted state;

computing a driver score, at a scoring module of the remote computing system, in response to generating the third output and based on the sensor measurements and the distracted state.

10. The method of claim 9, wherein transmitting the sensor measurements to the remote computing system is performed substantially instantaneously subsequent to determining that the driver of the vehicle is characterized by the distracted state.

11. The method of claim 9, further comprising determining a context of the distracted state, at the onboard system of the vehicle, based on the sensor measurements.

12. The method of claim 11, wherein the sensor measurements further comprise a second image of a region in front of the vehicle, and wherein determining the context of the distracted state comprises determining a following distance between a leading vehicle depicted in the second image and the vehicle.

13. The method of claim 11, wherein the context comprises a duration of the distracted state, wherein the distracted state comprises a difference between a gaze direction and a forward gaze direction, and further comprising determining a score of the distracted state based on the duration exceeding a threshold duration.

14. The method of claim 11, further comprising determining a score of the distracted state based on the context of the distracted state, wherein the score comprises a probability that the distraction state will lead to vehicle damage.

15. The method of claim 14, further comprising notifying the driver, at an output of the onboard system of the vehicle and in substantially real-time, based on the distracted state, the context of the distracted state, and the score of the distracted state.

16. The method of claim 14, wherein transmitting the sensor measurements to the remote computing system is performed in response to the score of the distracted state exceeding a threshold score.

17. The method of claim 9, further comprising transmitting a notification to a remote entity associated with the driver, in response to generating the third output, wherein the notification comprises the distracted state, a time point of distracted state determination, and the score.