System and method for responding to driver state

• US 10,759,436 B2
• Filed: 04/16/2019
• Issued: 09/01/2020
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Fung, Kin C., Dick, Timothy J.
• Primary Examiner(s): Jeanglaude, Gertrude Arthur
• Application Number:
  US16/385,108
• Publication Number:
  US 20190241190A1
• Time in Patent Office: 504 Days
• Field of Search: 701 36
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60R 25/25 using biometry
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/45 Detection of the body part being alive
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G07C 5/02 Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00
• CPC Class Codes: G07C 5/08 Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
• CPC Class Codes: G07C 9/00309 operated with bidirectional data transmission between data carrier and locks
• CPC Class Codes: G07C 9/00563 using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A method for controlling vehicle systems includes receiving monitoring information from one or more monitoring systems and determining a plurality of driver states based on the monitoring information from the one or more monitoring systems. The method includes determining a combined driver state based on the plurality of driver states and modifying control of one or more vehicle systems based on the combined driver state.

Claims

1. A method of controlling a vehicle, comprising:

receiving vehicle information from a low speed follow system;

detecting a hazard based on the vehicle information;

receiving monitoring information about a driver from a monitoring system, wherein the monitoring information includes head movement information about the driver and hand contact information about contact between the driver and a steering wheel of the vehicle;

determining a driver state based on the monitoring information and the hazard; and

modifying control of the vehicle based on the driver state.

2. The method of claim 1, wherein the hazard is a target vehicle in front of the vehicle, and determining the driver state is based on analyzing the monitoring information in relation to the target vehicle.

3. The method of claim 1, wherein modifying control of the vehicle includes modifying a control status of the low speed follow system based on the driver state.

4. The method of claim 3, wherein modifying control of the vehicle includes modifying control of the steering wheel to provide a visual indication based on the driver state and the control status.

5. The method of claim 2, wherein upon determining no contact between the driver and the steering wheel based on the hand contact information, modifying control of the vehicle includes modifying a control status of the low speed follow system to manual thereby controlling the vehicle with respect to the target vehicle using a manual input from the driver.

6. The method of claim 2, wherein upon determining at least one hand of the driver is in contact with the steering wheel based on the hand contact information and a head look of the driver is forward-looking based on the head movement information, modifying control of the vehicle includes modifying a control status of the low speed follow system to auto thereby controlling the vehicle automatically with respect to the target vehicle.

7. A system for controlling a vehicle, comprising:

a monitoring system for receiving monitoring information about a driver, wherein the monitoring information includes head movement information about the driver and hand contact information about contact between the driver and a steering wheel of the vehicle; and

an electronic control unit including a processor, wherein the processor;

receives vehicle information from a low speed follow system;

detects a hazard based on the vehicle information;

determines a driver state based on the monitoring information and the hazard; and

modifies control of the vehicle based on the driver state.

8. The system of claim 7, the processor detects the hazard as a target vehicle travelling in front of the vehicle based on the vehicle information and determines the driver state by analyzing the monitoring information in relation to the target vehicle.

9. The system of claim 7, wherein the processor modifies a control status of the low speed follow system based on the driver state.

10. The system of claim 9, wherein the processor modifies control of the steering wheel to provide a visual indication based on the driver state and the control status.

11. The system of claim 8, wherein upon the processor determining no contact between the driver and the steering wheel based on the hand contact information, the processor modifies a control status of the low speed follow system to operate in a manual mode and controls the vehicle with respect to the target vehicle based on manual input from the driver.

12. The system of claim 8, wherein upon the processor determining at least one hand of the driver is in contact with the steering wheel based on the hand contact information and a head look of the driver is forward-looking based on the head movement information, the processor modifies a control status of the low speed follow system to operate in an auto mode thereby controlling the vehicle automatically with respect to the target vehicle.

13. A method of controlling a vehicle, comprising:

receiving vehicle information from an automatic cruise control system;

detecting a hazard based on the vehicle information;

receiving monitoring information about a driver from a monitoring system, wherein the monitoring information includes head movement information about the driver and hand contact information about contact between the driver and a steering wheel of the vehicle;

determining a driver state based on the monitoring information and the hazard;

modifying control of the automatic cruise control system and a lane keep assist system based on the driver state.

14. The method of claim 13, wherein the hazard is a target vehicle in front of the vehicle, and determining the driver state is based on the hand contact information and analyzing the head movement information in relation to the target vehicle.

15. The method of claim 14, wherein upon determining at least one hand of the driver is in contact with the steering wheel based on the hand contact information, modifying control of the vehicle includes modifying a control status of the automatic cruise control system to manual thereby controlling a headway distance between the vehicle and the target vehicle using a manual input from the driver.

16. The method of claim 15, wherein upon determining a head look of the driver is forward-looking based on the head movement information modifying control of the vehicle includes modifying a control status of the lane keep assist system to manual, and upon determining the head look of the driver is non-forward-looking based on the head movement information, modifying control of the vehicle includes modifying the control status of the lane keep assist system to auto.

17. The method of claim 14, wherein upon determining no contact with the steering wheel based on the hand contact information, modifying the vehicle includes modifying a control status of the lane keep assist system to auto thereby controlling the vehicle automatically to drive within a current lane.

18. The method of claim 17, wherein modifying the vehicle includes modifying a gap setting of the automatic cruise control system based on the head movement information.

19. A system for controlling a vehicle, comprising:

a monitoring system for receiving monitoring information about a driver, wherein the monitoring information includes head movement information about the driver and hand contact information about contact between the driver and a steering wheel of the vehicle; and

an electronic control unit including a processor, wherein the processor;

receives vehicle information from an automatic cruise control system;

detects a hazard based on the vehicle information;

determines a driver state based on the monitoring information and the hazard; and

modifies control of the vehicle based on the driver state.

20. The system of claim 19, wherein the hazard is a target vehicle in front of the vehicle, and the processor determines the driver state is based on the hand contact information and analyzing the head movement information in relation to the target vehicle.

21. The system of claim 20, wherein upon the processor determining at least one hand of the driver is in contact with the steering wheel based on the hand contact information, the processor modifies a control status of the automatic cruise control system to manual and controls a headway distance between the vehicle and the target vehicle using a manual input from the driver.

22. The system of claim 21, wherein upon the processor determining a head look of the driver is forward-looking based on the head movement information the processor modifies a control status of a lane keep assist system to manual, and upon the processor determining the head look of the driver is non-forward-looking based on the head movement information, the processor modifies the control status the lane keep assist system to auto.

23. The system of claim 20, wherein upon the processor determining no contact with the steering wheel based on the hand contact information, the processor modifies a control status of a lane keep assist system to auto and controls the vehicle automatically to drive within a current lane.

24. The system of claim 23, wherein the processor modifies a gap setting of the automatic cruise control system based on the head movement information.

25. A method of controlling a vehicle, comprising:

receiving vehicle information from a lane departure warning system;

detecting a hazard based on the vehicle information;

receiving monitoring information about a driver from a monitoring system, wherein the monitoring information includes head movement information about the driver;

determining a driver state based on the monitoring information and the hazard; and

modifying control of the vehicle based on the driver state.

26. The method of claim 25, wherein the hazard is a lane deviation of the vehicle from a current lane.

27. The method of claim 26, wherein determining the driver state is based on the head movement information in relation to the vehicle or the lane deviation.

28. The method of claim 26, wherein upon determining a head look of the driver is a forward-looking head look in relation to the vehicle or the head look of the driver is directed in a same direction as the lane deviation, the driver state is determined to be attentive, and upon determining the head look of the driver is a non-forward-looking head look in relation to the vehicle or the head look of the driver is directed in a different direction in relation to the lane deviation, the driver state is determined to be inattentive.

29. The method of claim 28, wherein upon determining the driver state is attentive, modifying control of the vehicle includes modifying a control status of the lane departure warning system to suppress warnings emitted by the lane departure warning system.

30. The method of claim 28, wherein upon determining the driver state is in attentive, modifying control of the vehicle includes modifying a control status of the lane departure warning system to activate warnings emitted by the lane departure warning system.

31. A system for controlling a vehicle, comprising:

a monitoring system for receiving monitoring information about a driver, wherein the monitoring information includes head movement information about the driver; and

an electronic control unit including a processor, wherein the processor;

receives vehicle information from a lane departure warning system;

detects a hazard based on the vehicle information;

determines a driver state based on the monitoring information and the hazard; and

modifies control of the vehicle based on the driver state.

32. The system of claim 31, wherein the hazard is a lane deviation of the vehicle from a current lane.

33. The system of claim 32, wherein the processor determines the driver state based on the head movement information in relation to the vehicle or the lane deviation.

34. The system of claim 32, wherein the processor determines the driver state as attentive when a head look of the driver is a forward-looking head look in relation to the vehicle or the head look of the driver is directed in a same direction as the lane deviation, and the processor determines the driver state as inattentive when the head look of the driver is a non-forward-looking head look in relation to the vehicle or the head look of the driver is directed in a different direction in relation to the lane deviation.

35. The system of claim 34, wherein when the driver state is attentive, the processor modifies a control status of the lane departure warning system to ON thereby suppressing warnings emitted by the lane departure warning system.

36. The system of claim 34, wherein when the driver state in inattentive, the processor modifies a control status of the lane departure warning system to OFF thereby activating warnings emitted by the lane departure warning system.

37. A method of controlling a vehicle, comprising:

receiving vehicle information from a blind spot indicator system;

detecting a hazard based on the vehicle information;

receiving monitoring information about a driver from a monitoring system, wherein the monitoring information includes head movement information;

determining a driver state based on the monitoring information and the hazard;

modifying control of the vehicle based on the driver state.

38. The method of claim 37, wherein the hazard is an object in a blind spot monitoring zone of the vehicle.

39. The method of claim 38, wherein determining the driver state is based on the head movement information in relation to the object or the blind spot monitoring zone.

40. The method of claim 37, wherein determining the driver state includes determining whether the driver is aware of the hazard based on the head movement information.

41. The method of claim 37, wherein the driver state is determined to be attentive when the head movement information indicates the driver is looking at the hazard.

42. The method of claim 41, wherein when the driver state is attentive, modifying control of the vehicle includes modifying a control status of the blind spot indicator system to suppress warnings emitted by the blind spot indicator system.

43. A system for controlling a vehicle, comprising:

a monitoring system for receiving monitoring information about a driver, wherein the monitoring information includes head movement information about the driver; and

an electronic control unit including a processor, wherein the processor;

receives vehicle information from a blind spot indicator system;

detects a hazard based on the vehicle information;

determines a driver state based on the monitoring information and the hazard; and

modifies control of the vehicle based on the driver state.

44. The system of claim 43, wherein the hazard is an object detected by the blind spot indicator system in a blind spot monitoring zone of the vehicle.

45. The system of claim 44, wherein the processor determines the driver state by analyzing the head movement information relative to the object or the blind spot monitoring zone.

46. The system of claim 43, wherein the processor determines whether the driver is aware of the hazard based on the head movement information relative to the hazard.

47. The system of claim 43, wherein the processor determines the driver state is attentive when the processor determines the driver is looking at the hazard based on the head movement information.

48. The system of claim 47, wherein when the driver state is attentive, the processor modifies a control status of the blind spot indicator system to suppress warnings emitted by the blind spot indicator system.