Detecting risky driving with machine vision

• US 9,996,756 B2
• Filed: 08/31/2015
• Issued: 06/12/2018
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: Lytx, Inc.
• Original Assignee: Lytx, Inc.
• Inventors: Hoye, Brett, Surpi, Gabriela
• Primary Examiner(s): CONNER, SEAN M
• Application Number:
  US14/841,376
• Publication Number:
  US 20170061222A1
• Time in Patent Office: 1,016 Days
• Field of Search: None
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/06 in graphical form
• CPC Class Codes: G07C 5/0825 using optical means
• CPC Class Codes: G07C 5/0891 Video recorder in combination with video camera

Abstract

A system for driver risk determination using machine vision includes an interface and one or more processors. The interface is to receive one or more video data streams. The one or more processors is/are to determine, whether a driving behavior risk type appears in one of the one or more video data streams; and in the event that the driving behavior risk type appears in the one of the one or more video data streams, indicate the driver behavior risk type.

Claims

1. A system for driver risk determination using machine vision, comprising:

an interface to receive one or more video data streams of a vehicle, wherein the one or more video data streams comprise one or more of a video stream from an interior camera of the vehicle and a video stream from an exterior camera of the vehicle;

one or more processors to;

determine, using machine vision, whether a driving behavior risk type appears in one of the one or more video data streams;

in response to the driving behavior risk type appearing in the one of the one or more video data streams;

indicate an incidence of a driver behavior risk type;

determine a change in a driver risk based at least on a comparison of a number of incidences of driver risk types over a first period of time and a number of incidences of driver risk types over a second period of time, wherein the number of incidences of driver risk types over the second period of time includes the incidence of the driver behavior risk type, wherein the first period of time consists of N units of time, wherein the second period of time consists of a most recent 1/N units of time, and wherein the first period of time includes the second period of time; and

determine that the change in driver risk comprises an increase in risk when the number of incidences of driver risk types over the second period of time exceeds the number of incidences of driver risk types over the first period of time divided by N.

2. The system of claim 1, wherein the processor is further to:

determine an incident severity of driver behavior risk based at least in part on the driver behavior risk type.

3. The system of claim 2, wherein the incident severity of driver behavior risk is determined based at least in part on a fraction of time the behavior was sustained.

4. The system of claim 2, wherein the incident severity of driver behavior risk is determined based at least in part on how many driver behavior risk types co-occur in the incident.

5. The system of claim 2, wherein one processor of the one or more processors is further to:

in response to the incident severity of driver behavior risk being above a driver behavior risk threshold, provide the driver behavior risk type to a server.

6. The system of claim 2, wherein one processor of the one or more processors is further to:

in response to the incident severity of driver behavior risk being above a driver behavior risk threshold, store video data from the one of the one or more video data streams.

7. The system of claim 2, wherein one processor of the one or more processors is further to:

in response to the incident severity of driver behavior risk being above a driver behavior risk threshold, provide driver behavior risk metadata and the one of the one or more video data streams to a server.

8. The system of claim 7, wherein the server is used to determine a past or a future driver risk based at least in part on the driver behavior risk metadata or the one of the one or more video data streams.

9. The system of claim 8, wherein the past or the future driver risk is based at least in part on a behavior observed during a human review of the one of the one or more video data streams.

10. The system of claim 8, wherein the past or the future driver risk is determined based at least in part on a number of driver behavior events of the driver behavior risk type.

11. The system of claim 8, wherein the past or the future driver risk is determined based at least in part on a number of driver behavior events of a plurality of driver behavior risk types.

12. The system of claim 8, wherein the past or the future driver risk is based at least in part on a number of driver behavior events of the driver behavior risk type occurring during a recent period of time.

13. The system of claim 8, wherein the past or the future driver risk is based at least in part on a number of driver behavior events of a plurality of driver behavior risk types occurring during recent periods of time.

14. The system of claim 8, wherein the past or the future driver risk is based at least in part on a severity of a determined driving behavior risk incidence.

15. The system of claim 8, wherein the past or the future driver risk is based at least in part on a severity of a determined driving behavior risk type.

16. The system of claim 1, wherein one processor of the one or more processors is further to:

in response to a driver risk being above a threshold, indicate a coachable event.

17. The system of claim 1, wherein the one of the one or more video data streams comprises the interior camera and the driver behavior risk type is determined based at least in part on a gaze direction.

18. The system of claim 17, wherein the gaze direction comprises a gaze not directed at the road ahead.

19. The system of claim 1, wherein the one of the one or more video data streams comprises the interior camera and the driver behavior risk type is determined based at least in part on a head orientation.

20. The system of claim 19, wherein the head orientation comprises a direction not pointing at the road ahead.

21. The system of claim 1, wherein the one of the one or more video data streams comprises the interior camera and the driver behavior risk type is determined based at least in part on a body position.

22. The system of claim 1, wherein the one of the one or more video data streams comprises the interior camera and the driver behavior risk type is determined based at least in part on a hand position.

23. The system of claim 22, wherein the hand position comprises a hand position holding a phone.

24. The system of claim 22, wherein the hand position comprises a hand position during eating.

25. The system of claim 1, wherein the one of the one or more video data streams comprises the interior camera and the driver behavior risk type is determined based at least in part on an eyelid position.

26. The system of claim 25, wherein the eyelid position comprises an eyelid position of closed eyes.

27. The system of claim 1, wherein the one of the one or more video data streams comprises the exterior camera and the driver behavior risk type is determined based at least in part on a lane drift.

28. The system of claim 1, wherein the one of the one or more video data streams comprises the exterior camera and the driver behavior risk type is determined based at least in part on a lane choice.

29. The system of claim 1, wherein the one of the one or more video data streams comprises the exterior camera and the driver behavior risk type is determined based at least in part on a turn indicator of the vehicle.

30. The system of claim 1, wherein the one of the one or more video data streams comprises the exterior camera and the driver behavior risk type is determined based at least in part on a speed relative to a traffic speed.

31. The system of claim 1, wherein the one of the one or more video data streams comprises the exterior camera and the driver behavior risk type is determined based at least in part on a lane change frequency.

32. The system of claim 1, wherein the one or more video data streams comprises one or more of the following:

a video stream from a vehicle camera, or a video stream from the interior camera and a video stream from the exterior camera.

35. The system of claim 1, wherein the interface comprises an on-board diagnostics (OBD) bus.

33. A method for driver risk determination using machine vision, comprising:

receiving one or more video data streams of a vehicle, the one or more video data streams comprising one or more of a video stream from an interior camera of the vehicle and a video stream from an exterior camera of the vehicle;

determining, using one or more processors, whether a driving behavior risk type appears in one of the one or more video data streams; and

in response to a driving behavior risk type appearing in the one of the one or more video data streams;

indicating an incidence of a driver behavior risk type;

determining a change in a driver risk based at least on a comparison of a number of incidences of driver risk types over a first period of time and a number of incidences of driver risk types over a second period of time, wherein the number of incidences of driver risk types over the second period of time includes the indicated incidence of the driver behavior risk type, wherein the first period of time consists of N units of time, wherein the second period of time consists of a most recent 1/N units of time, and wherein the first period of time includes the second period of time; and

determining that the change in driver risk comprises an increase in risk when the number of incidences of driver risk types over the second period of time exceeds the number of incidences of driver risk types over the first period of time divided by N.

34. A computer program product for driver risk determination using machine vision, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving one or more video data streams of a vehicle, the one or more video data streams comprising one or more of a video stream from an interior camera of the vehicle and a video stream from an exterior camera of the vehicle;

determining, using machine vision, whether a driving behavior risk type appears in one of the one or more video data streams; and

in response to a driving behavior risk type appearing in the one of the one or more video data streams;

indicating an incidence of a driver behavior risk type;

determining a change in a driver risk based at least on a comparison of a number of incidences of driver risk types over a first period of time and a number of incidences of driver risk types over a second period of time, wherein the number of incidences of driver risk types over the second period of time includes the indicated incidence of the driver behavior risk type, wherein the first period of time consists of N units of time, wherein the second period of time consists of a most recent 1/N units of time, and wherein the first period of time includes the second period of time; and

determining that the change in driver risk comprises an increase in risk when the number of incidences of driver risk types over the second period of time exceeds the number of incidences of driver risk types over the first period of time divided by N.