System and method for responding to driver state

• US 10,780,891 B2
• Filed: 05/22/2019
• Issued: 09/22/2020
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Fung, Kin C., Dick, Timothy J.
• Primary Examiner(s): Jeanglaude, Gertrude Arthur
• Application Number:
  US16/419,161
• Publication Number:
  US 20190276034A1
• Time in Patent Office: 489 Days
• Field of Search: 701 36
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60R 25/25 using biometry
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/45 Detection of the body part being alive
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G07C 5/02 Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00
• CPC Class Codes: G07C 5/08 Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
• CPC Class Codes: G07C 9/00309 operated with bidirectional data transmission between data carrier and locks
• CPC Class Codes: G07C 9/00563 using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A method for controlling vehicle systems includes receiving monitoring information from one or more monitoring systems and determining a plurality of driver states based on the monitoring information from the one or more monitoring systems. The method includes determining a combined driver state based on the plurality of driver states and modifying control of one or more vehicle systems based on the combined driver state.

Claims

1. A method of controlling a vehicle, comprising:

receiving monitoring information from a monitoring system;

receiving vehicle information from a vehicle system, wherein the vehicle information includes an auto control status of the vehicle system;

determining a driver state based on the monitoring information; and

modifying a visual device based on the driver state and the auto control status of the vehicle system.

2. The method of claim 1, wherein the visual device is a light bar integrated with a steering wheel.

3. The method of claim 1, wherein modifying the visual device includes modifying a light element of the visual device according to the driver state and the auto control status thereby providing an indication of the driver state and the auto control status.

4. The method of claim 1, wherein the monitoring information includes at least one of head movement information, eye gaze information, and hand contact information.

5. The method of claim 4, wherein determining the driver state includes determining a first driver state based on the head movement information, a second driver state based on the eye gaze information, a third driver state based on the hand contact information, and a vehicular state based on the auto control status.

6. The method of claim 5, wherein determining the driver state includes determining a combined driver state based on the first driver state, the second driver state, and the third driver state, and modifying the visual indicator of the vehicle based on the combined driver state and the vehicular state.

7. The method of claim 1, including upon determining the driver is distracted based on the driver state, and the auto control status is ON, modifying the visual device to increase alertness of the driver and provide an indication of the auto control status.

8. The method of claim 7, including modifying the visual device to emit a red colored light according to the driver state and the auto control status.

9. The method of claim 1, upon determining the driver is not distracted based on the driver state, and the auto control status is ON, modifying the visual device to emit a green colored light.

10. The method of claim 1, upon determining the driver is not distracted based on the driver state, and the auto control status is PARTIAL, modifying the visual device includes modifying the visual device to emit a partially green colored light.

11. The method of claim 1, upon determining the auto control status is OFF, modifying the visual device includes modifying the visual device to provide an indication that the driver is manually controlling the vehicle system.

12. The method of claim 1, including receiving an auto control mode of the vehicle, and modifying the visual device based on the driver state, the auto control status of the vehicle system, and the auto control mode of the vehicle.

13. A system for controlling a vehicle, comprising:

a monitoring system; and

an electronic control unit including a processor, wherein the processor;

receives monitoring information from the monitoring system;

receives vehicle information from a vehicle system, wherein the vehicle information includes an auto control status of the vehicle system;

determines a driver state based on the monitoring information; and

modifies a visual device based on the driver state and the auto control status of the vehicle system.

14. The system of claim 13, wherein the visual device is a light bar integrated with a steering wheel.

15. The system of claim 13, wherein the visual device includes a light element and the processor activates the light element according to the driver state and the auto control status thereby providing an indication of the driver state and the auto control status.

16. The system of claim 13, wherein the monitoring information includes at least one of head movement information from a head movement monitoring system, eye gaze information from an eye gaze monitoring information, and hand contact information from a steering wheel system.

17. The system of claim 16, wherein the processor determines a first driver state based on the head movement information, a second driver state based on the eye gaze information, a third driver state based on the hand contact information, and a vehicular state based on the auto control status.

18. The system of claim 17, wherein the processor determines a combined driver state based on the first driver state, the second driver state, and the third driver state, and the processor modifies the visual indicator of the vehicle based on the combined driver state and the vehicular state.

19. The system of claim 13, wherein when the processor determines the driver state is distracted and the auto control status is ON, the processor activates the visual device to increase alertness of the driver and provide an indication of the auto control status.

20. The system of claim 19, including the processor modifying a control parameter of the visual device thereby emitting a red colored light according to the driver state and the auto control status.

21. The system of claim 13, wherein when the processor determines the driver is not distracted based on the driver state, and the processor determines the auto control status is ON, the processor activates the visual device to emit a green colored light.

22. The system of claim 13, wherein when the processor determines the driver is not distracted based on the driver state, and the processor determines the auto control status is PARTIAL, the processor activates the visual device to emit a partially green colored light.

23. The system of claim 13, wherein when the processor determines the auto control status is OFF, the processor modifies a control parameter of the visual device and activates the visual device using the control parameter to provide an indication that the driver is manually controlling the vehicle system.

24. The system of claim 13, including the processor receiving an auto control mode of the vehicle, and the processor modifies the visual device based on the driver state, the auto control status of the vehicle system, and the auto control mode of the vehicle.