DETECTION OF DRIVER BEHAVIORS USING IN-VEHICLE SYSTEMS AND METHODS

• US 20160046298A1
• Filed: 08/18/2015
• Published: 02/18/2016
• Est. Priority Date: 08/18/2014
• Status: Active Grant

• Current Assignee: Trimble Navigation Limited (Trimble Inc.)
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: DeRuyck, Jason, McLaughlin, Brian Richard
• Granted Patent: US 9,714,037 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 2050/0029 Mathematical model of the driver
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 2540/30 Driving style
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 2710/30 Auxiliary equipments
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

A system includes a motion sensor system configured for deployment in a cab of a vehicle and to generate substantially in real-time a digital mapping of driver movement during operation of the vehicle. A computer is configured for deployment in the cab and coupled to the motion sensor system. A driver behavior detector is coupled to the computer and configured to detect a driver distraction event using the driver movement mapping. A communication device at the vehicle is configured to communicate an alert message to one or both of a user interface device in the cab and a remote system in response to the detected driver distraction event.

Claims

1. A system, comprising:

a motion sensor system configured for deployment in a cab of a vehicle and to generate substantially in real-time a digital mapping of driver movement during operation of the vehicle;

an audio sensor configured for deployment in the cab and configured to generate an audio signal responsive to driver movement during operation of the vehicle;

a computer configured for deployment in the cab and coupled to the motion sensor system and the audio sensor;

a driver behavior detector coupled to the computer and configured to detect a driver distraction event using the driver movement mapping and the audio signal; and

a communication device at the vehicle and configured to communicate an alert message to one or both of a user interface device in the cab and a remote system in response to the detected driver distraction event.

2. The system of claim 1, wherein the motion sensor system comprises a three-dimensional motion sensor configured to generate substantially in real-time the digital mapping of the driver'"'"'s movement.

3. The system of claim 1, wherein the motion sensor system comprises:

an infrared projector or emitter;

an infrared sensor; and

a camera.

4. The system of claim 3, wherein the camera comprises an RGB color camera.

5. The system of claim 1, wherein the audio sensor comprises a multi-array microphone.

6. The system of claim 1, wherein the computer is configured to:

detect a potential driver distraction event using the digital mapping; and

determine that the potential driver distraction event is an actual driver distraction event using the audio signal.

7. The system of claim 6, wherein the computer is configured to determine a contextual characteristic of the audio signal, and determine if the contextual characteristic is consistent with distracted driver activity.

8. The system of claim 1, wherein the motion sensor system is configured to generate a three-dimensional contour or skeletal outline of the driver'"'"'s face.

9. The system of claim 1, wherein:

the motion sensor system is configured to detect changes of a retina of the driver; and

the computer is configured to detect the driver distraction event using detected changes of the driver'"'"'s retina.

10. The system of claim 1, wherein the alert message communicated to the remote system comprises an informational message devoid of photographic or video data.

11. The system of claim 1, wherein the alert message communicated to the remote system comprises an informational message and at least some facial mapping data.

12. The system of claim 1, further comprising an event detector configured to detect vehicle events based on one or both of vehicle computer or bus data and one or more sensors mounted at the vehicle, wherein:

the computer is configured to detect the driver distraction event using the facial mapping, the audio signal, and an output produced by the event detector.

13. The system of claim 1, wherein:

the audio sensor is configured to operate in a listening mode;

the computer is configured to generate a trigger signal in response to an output of the audio sensor while operating in the listening mode; and

the motion sensor system is configured to generate substantially in real-time the digital mapping of the driver'"'"'s face in response to the trigger signal.

14. The system of claim 1, further comprising a forward-looking motion sensor system configured for deployment in the cab and configured to sense substantially in real-time movement of a person located in front of the vehicle while the vehicle is in park.

15. The system of claim 1, wherein:

output from at least the motion sensor system is stored in a memory deployed at the cab; and

the computer is configured to develop a profile or model for a specific driver of the vehicle using at least the stored motion sensor system output.

16. The system of claim 15, wherein the computer is configured to refine the profile or model for the specific driver in response to acquisition of data subsequently generated by one or both of the motion sensor system and the driver behavior detector.

17. The system of claim 1, wherein the computer is configured to:

access, from an in-cab memory, an expected behavior profile or model developed from previously acquired digital mappings of the driver'"'"'s movements; and

compare the digital mapping of driver movement to the expected behavior profile or model to determine whether a potential driver distraction event is an actual driver distraction event.

18. The system of claim 1, wherein the computer is configured to:

compare the digital mapping of driver movement to an expected behavior profile or model developed from previously acquired digital mappings of the driver'"'"'s movements;

count repeated occurrences where the digital mapping of driver movement matches the expected behavior profile or model;

compare the count to a threshold; and

determine that a potential driver distraction event is an actual driver distraction event in response to the count meeting or exceeding the threshold.

19. The system of claim 1, wherein the computer is configured to:

compare the digital mapping of driver movement to an expected behavior profile or model developed from previously acquired digital mappings of the driver'"'"'s movements;

measure the duration of repeated occurrences where the digital mapping of driver movement matches the expected behavior profile or model;

compare the measured duration to a threshold; and

determine that a potential driver distraction event is an actual driver distraction event in response to the measured duration meeting or exceeding the threshold.

20. The system of claim 1, wherein the driver behavior detector is configured to:

receive additional data comprising one or more of vehicle computer data, vehicle sensor data, and environmental data; and

detect a driver distraction event using the driver movement mapping, the audio signal, and the additional data.

21. A system, comprising:

an imaging system configured for deployment in a cab of a vehicle and to generate substantially in real-time an image of driver movement during operation of the vehicle;

an audio sensor configured for deployment in the cab and configured to generate an audio signal responsive to driver movement during operation of the vehicle;

a computer configured for deployment in the cab and coupled to the imaging system and the audio sensor;

a driver behavior detector coupled to the computer and configured to detect a driver distraction event using the driver movement image and the audio signal; and

a communication device at the vehicle and configured to communicate an alert message to one or both of a user interface device in the cab and a remote system in response to the detected driver distraction event.

22. A method, comprising:

generating substantially in real-time a digital mapping of driver movement during operation of a vehicle;

generating an audio signal responsive to driver movement during operation of the vehicle;

detecting a driver distraction event using the driver movement mapping and the audio signal; and

communicating an alert message to one or both of a user interface device in the vehicle and a remote system in response to the detected driver distraction event.