Real-time driver monitoring and feedback reporting system

• US 9,373,203 B1
• Filed: 09/23/2014
• Issued: 06/21/2016
• Est. Priority Date: 09/23/2014
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Fields, Brian Mark, Roberson, Steve, Harish, Abhishek, Lim, Hyunji, Waughtel, Matthew James
• Primary Examiner(s): Beaulieu, Yonel
• Assistant Examiner(s): Kleinman, Lail
• Application Number:
  US14/494,251
• Time in Patent Office: 637 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/0816 Indicating performance data, e.g. occurrence of a malfunction
• CPC Class Codes: G07C 5/0858 wherein the data carrier is removable
• CPC Class Codes: G07C 5/12 in graphical form
• CPC Class Codes: G09B 19/14 Traffic procedures, e.g. traffic regulations
• CPC Class Codes: G09B 19/167 Control of land vehicles
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/40 for vehicles, e.g. vehicle-to-pedestrians [V2P]
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

Real-time feedback may be generated during a driving session from status data collected via one or more sensors. This feedback data may include various metrics related to a student'"'"'s driving skills, which may be utilized to generate tagged driving events. A user may also manually enter tagged driving events. The tagged driving events may include real-time information based on the collected status data, such as acceleration, braking, cornering data, descriptions entered by the user. The location of these tagged driving events may be mapped in real time. Sensors used to collect this data may be integrated within the device displaying the feedback or located on another device, in which case the status data may be communicated to the device displaying the feedback. The displayed real-time feedback may be viewed by an instructor to assess a student'"'"'s driving skills in real time and to calculate a driver feedback score.

Claims

1. A computer-implemented method for displaying real-time data during a driving session on a first mobile computing device located in a vehicle, comprising:

receiving, at the first mobile computing device by one or more processors, a plurality of status metrics in real-time from a second mobile computing device located in the vehicle during the driving session, the plurality of status metrics being generated by the second mobile computing device and including data indicative of one or more of (i) the vehicle'"'"'s acceleration, (ii) the vehicle'"'"'s movement while braking, and (iii) the vehicle'"'"'s movement while cornering,

wherein the second mobile computing device includes a second display configured to display driving session information based upon the plurality of status metrics, and

wherein the first mobile computing device includes a first display configured to concurrently display a larger amount of different types of driving session information based upon the plurality of status metrics than the different types of driving session information that could be concurrently displayed via the second display;

generating, at the first mobile computing device by one or more processors, a tagged driving event in real-time when one of the plurality of status metrics meets or exceeds a respective status metric threshold; and

concurrently displaying, on the first display, the driving session information including (i) a map showing a route driven by the vehicle during the driving session and the tagged driving event on the map in real-time at a location where the tagged driving event occurred, (ii) a real-time driver status, and (iii) a real-time vehicle status.

2. The method of claim 1, wherein the real-time driver status comprises one or more of:

driver eye movement;

a direction of a driver'"'"'s gaze;

driver head position;

or

a driver biometric.

3. The method of claim 1, wherein the real-time vehicle status comprises one or more of:

vehicle road speed;

vehicle engine speed;

a time to collision (CTT);

a lane position of the vehicle;

vehicle light status;

or

vehicle braking status.

4. The method of claim 1, wherein the tagged driving event is from among a plurality of tagged driving events that occur during the driving session, and further comprising:

calculating, at the first mobile computing device by one or more processors, a severity level for each of the plurality of tagged driving events based upon an amount by which each of the plurality of tagged driving events exceeds a respective status metric threshold; and

calculating, at the first mobile computing device by one or more processors, a driver feedback score after the driving session has ended based on the severity level of each of the plurality of tagged driving events.

5. The method of claim 1, wherein the status metric threshold corresponds to one or more of:

a threshold acceleration status metric value;

a threshold cornering status metric value; and

a threshold braking status metric value.

6. The method of claim 1,

wherein the plurality of status metrics received in real-time from the second mobile computing device are utilized to indicate the driver status, and further comprising:

receiving a second plurality of status metrics in real-time from an on-board computer associated with the vehicle, the second plurality of status metrics being utilized to indicate the vehicle status.

7. The method of claim 1, further comprising:

receiving, at the first mobile computing device by one or more processors, a user input indicative of a driving event during the driving session;

determining, at the first mobile computing device by one or more processors, a location of the vehicle where the driving event occurred at a time corresponding to when the user input was received; and

displaying, on the first display, a custom tagged driving event on the map at the location of the vehicle where the driving event occurred in real-time.

22. The method of claim 1, wherein:

the real-time driver status includes a gaze location chart that provides an indication of a direction of the driver'"'"'s gaze in real-time during the driving session,

the real-time vehicle status includes a speed indicator that provides an indication of a current vehicle speed in real-time during the driving session, and

the act of concurrently displaying the driving session information includes concurrently displaying (i) the map showing a route driven by the vehicle during the driving session and the tagged driving event on the map in real-time at a location where the tagged driving event occurred, (ii) the gaze location chart, and (iii) the speed indicator.

8. A non-transitory, tangible computer-readable medium storing machine readable instructions for displaying real-time data during a driving session on a first mobile computing device located in a vehicle, that when executed by a processor, cause the processor to:

receive a plurality of status metrics in real-time from a second mobile computing device located in the vehicle during the driving session, the plurality of status metrics being generated by the second mobile computing device and including data indicative of one or more of (i) the vehicle'"'"'s acceleration, (ii) the vehicle'"'"'s movement while braking, and (iii) the vehicle'"'"'s movement while cornering,

wherein the second mobile computing device includes a second display configured to display driving session information based upon the plurality of status metrics, and

wherein the first mobile computing device includes a first display configured to concurrently display a larger amount of different types of driving session information based upon the plurality of status metrics than the different types of driving session information that could be concurrently displayed via the second display;

generate a tagged driving event in real-time when one of the plurality of status metrics meets or exceeds a respective status metric threshold; and

concurrently display, on the first display, the driving session information including (i) a map showing a route driven by the vehicle during the driving session and the tagged driving event on the map in real-time at a location where the tagged driving event occurred, (ii) a real-time driver status, and (iii) a real-time vehicle status.

9. The non-transitory, tangible computer-readable medium of claim 8, wherein the real-time driver status comprises one or more of:

driver eye movement;

a direction of a driver'"'"'s gaze;

driver head position;

or

a driver biometric.

10. The non-transitory, tangible computer-readable medium of claim 8, wherein the real-time vehicle status comprises one or more of:

vehicle road speed;

vehicle engine speed;

time to collision (CTT);

a lane position of the vehicle;

vehicle light status;

or

vehicle braking status.

11. The non-transitory, tangible computer-readable medium of claim 8, wherein the tagged driving event is from among a plurality of tagged driving events that occur during the driving session, and further including instructions, that when executed by the processor, cause the processor to:

calculate a severity level for each of the plurality of tagged driving events based upon an amount by which each of the plurality of tagged driving events exceeds a respective status metric threshold; and

calculate a driver feedback score after the driving session has ended based on the severity level of each of the plurality of tagged driving events.

12. The non-transitory, tangible computer-readable medium of claim 8, wherein the status metric threshold corresponds to one or more of:

a threshold acceleration status metric value;

a threshold cornering status metric value; and

a threshold braking status metric value.

13. The non-transitory, tangible computer-readable medium of claim 8, wherein the plurality of status metrics received in real-time from the second mobile computing device are utilized to indicate the driver status, and further including instructions, that when executed by the processor, cause the processor to

receive a second plurality of status metrics in real-time from an on-board computer associated with the vehicle, the second plurality of status metrics being utilized to indicate the vehicle status.

14. The non-transitory, tangible computer-readable medium of claim 9, further including instructions, that when executed by the processor, cause the processor to:

receive a user input indicative of a driving event during the driving session;

determine a location of the vehicle where the driving event occurred at a time corresponding to when the user input was received; and

display, on the first display, a custom tagged driving event on the map at the location of the vehicle where the driving event occurred.

23. The non-transitory, tangible computer-readable medium of claim 8, wherein:

the real-time driver status includes a gaze location chart that provides an indication of a direction of the driver'"'"'s gaze in real-time during the driving session,

the real-time vehicle status includes a speed indicator that provides an indication of a current vehicle speed in real-time during the driving session, and

wherein the instructions to concurrently display the driving session information further include instructions that, when executed by the processor, cause the processor to concurrently display (i) the map showing a route driven by the vehicle during the driving session and the tagged driving event on the map in real-time at a location where the tagged driving event occurred, (ii) the gaze location chart, and (iii) the speed indicator.

15. A first mobile computing device for displaying real-time data during a driving session when located in a vehicle, comprising:

a processor configured to;

receive a plurality of status metrics in real-time from a second mobile computing device located in the vehicle during the driving session, the plurality of status metrics being generated by the second mobile computing device and including data indicative of one or more of (i) the vehicle'"'"'s acceleration, (ii) the vehicle'"'"'s movement while braking, and (iii) the vehicle'"'"'s movement while cornering; and

generate a tagged driving event in real-time when one of the plurality of status metrics meets or exceeds a respective status metric threshold,

wherein the second mobile computing device includes a second display configured to display driving session information based upon the plurality of status metrics, and

a first display configured to concurrently display a larger amount of different types of driving session information based upon the plurality of status metrics than the different types of driving session information that could be concurrently displayed via the second display; and

a graphical processing unit (GPU) configured to cause the first display to concurrently display the driving session information including (i) a map showing a route driven by the vehicle during the driving session and the tagged driving event on the map in real-time at a location where the tagged driving event occurred, (ii) a real-time driver status, and (iii) a real-time vehicle status.

16. The first mobile computing device of claim 15, wherein the real-time driver status comprises one or more of:

driver eye movement;

a direction of a driver'"'"'s gaze;

driver head position;

or

a driver biometric.

17. The first mobile computing device of claim 15, wherein the real-time vehicle status comprises one or more of:

vehicle road speed;

vehicle engine speed;

time to collision (CTT);

a lane position of the vehicle;

vehicle light status;

or

vehicle braking status.

18. The first mobile computing device of claim 15, wherein the tagged driving event is from among a plurality of tagged driving events that occur during the driving session, and

wherein the processor is further configured to:

calculate a severity level for each of the plurality of tagged driving events based upon an amount by which each of the plurality of tagged driving events exceeds a respective status metric threshold; and

calculate a driver feedback score after the driving session has ended based on the severity level of each of the plurality of tagged driving events.

19. The first mobile computing device of claim 15, wherein the status metric threshold corresponds to one or more of:

a threshold acceleration status metric value;

a threshold cornering status metric value; and

a threshold braking status metric value.

20. The first mobile computing device of claim 15, wherein the plurality of status metrics received in real-time from the second mobile computing device are utilized to indicate the driver status, and

wherein the processor is further configured to receive a second plurality of status metrics in real-time from an on-board computer associated with the vehicle, the second plurality of status metrics being utilized to indicate the vehicle status.

21. The first mobile computing device of claim 15, wherein the processor is further configured to:

receive a user input at the first mobile computing device indicative of a driving event during the driving session; and

determine a location of the vehicle where the driving event occurred at a time corresponding to when the user input was received, and

and wherein the GPU is further configured to display, on the first display, a custom tagged driving event on the map at the location of the vehicle where the driving event occurred.

24. The first mobile computing device of claim 15, wherein:

the real-time driver status includes a gaze location chart that provides an indication of a direction of the driver'"'"'s gaze in real-time during the driving session,

the real-time vehicle status includes a speed indicator that provides an indication of a current vehicle speed in real-time during the driving session, and

the GPU is further configured to cause the first display to concurrently display (i) the map showing a route driven by the vehicle during the driving session and the tagged driving event on the map in real-time at a location where the tagged driving event occurred, (ii) the gaze location chart, and (iii) the speed indicator.