System and method for responding to driver state

• US 9,751,534 B2
• Filed: 09/11/2015
• Issued: 09/05/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Fung, Kin C., Dick, Timothy J.
• Primary Examiner(s): ARTHUR JEANGLAUD, GERTRUDE
• Application Number:
  US14/851,753
• Publication Number:
  US 20160001781A1
• Time in Patent Office: 725 Days
• Field of Search: 701 36, 701 70, 3404255, 340436, 340438, 340439, 180271
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60R 25/25 using biometry
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/45 Detection of the body part being alive
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G07C 5/02 Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00
• CPC Class Codes: G07C 5/08 Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
• CPC Class Codes: G07C 9/00309 operated with bidirectional data transmission between data carrier and locks
• CPC Class Codes: G07C 9/00563 using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A method for controlling vehicle systems includes receiving monitoring information from one or more monitoring systems and determining a plurality of driver states based on the monitoring information from the one or more monitoring systems. The method includes determining a combined driver state based on the plurality of driver states and modifying control of one or more vehicle systems based on the combined driver state.

Claims

1. A method of controlling vehicle systems in a motor vehicle, comprising:

receiving monitoring information from one or more monitoring systems;

determining a plurality of driver states of a driver based on the monitoring information from the one or more monitoring systems;

determining a combined driver state index based on the plurality of driver states, wherein the combined driver state index is a variable dependent on a result of evaluating each of the plurality of driver states together; and

modifying control of one or more vehicle systems as a function of the combined driver state index.

2. The method of claim 1, wherein determining the combined driver state index is based on a first driver state selected from the plurality of driver states and a second driver state selected from the plurality of driver states.

3. The method of claim 2, wherein determining the combined driver state index is further based on a third driver state selected from the plurality of driver states.

4. The method of claim 3, wherein the plurality of driver states being at least one of the following driver state types:

a physiological driver state, a behavioral driver state, or a vehicular-sensed driver state.

5. The method of claim 4, wherein the physiological driver state is based on physiological information, the behavioral driver state is based on behavioral information, and the vehicular-sensed driver state is based on vehicle information.

6. The method of claim 4, wherein the first driver state selected from the plurality of driver states is a physiological driver state, the second driver state selected from the plurality of driver states is a behavioral driver state, and the third driver state selected from the plurality of driver states is a vehicular-sensed driver state.

7. The method of claim 4, wherein the first driver state selected from the plurality of driver states and the second driver state selected from the plurality of driver states are the same driver state type.

8. The method of claim 4, wherein the first driver state is a physiological driver state and the second driver state is a behavioral driver state.

9. The method of claim 8, wherein the one or more monitoring systems include a sensor for receiving physiological information and behavioral information, and the physiological driver state is based on the physiological information and the behavioral driver state is based on the behavioral information received from the sensor.

10. The method of claim 9, wherein the physiological information is heart rate information and the behavioral information is head movement information.

11. The method of claim 9, wherein the sensor is an optical sensor for receiving the physiological information and the behavioral information.

12. The method of claim 3, wherein determining a combined driver state index further includes determining if the combined driver state index indicates a distracted driver state or a non-distracted driver state based on the plurality of driver states.

13. The method of claim 12, wherein upon determining at least one of the first driver state selected from the plurality of driver states state or the second driver state selected from the plurality of driver states indicates a distracted driver state, the combined driver state index is determined to indicate the distracted driver state.

14. The method of claim 12, wherein upon determining at least one of the first driver state selected from the plurality of driver states or the second driver state selected from the plurality of driver states indicates a non-distracted driver state, the combined driver state index is determined to indicate the non-distracted driver state.

15. The method of claim 14, further including upon determining the third driver state selected from the plurality of driver states indicates a distracted driver state, the combined driver state index is determined to indicate the distracted driver state.

16. A method of controlling vehicle systems in a motor vehicle, comprising:

receiving monitoring information from one or more monitoring systems;

determining a plurality of driver states selected from one of the following types of driver states;

physiological driver state, behavioral driver state, and vehicular-sensed driver state, wherein the plurality of driver states are based on the monitoring information from the one or more monitoring systems;

determining a combined driver state index based on the plurality of driver states, wherein the combined driver state index is a physiological, a behavioral, and a vehicular-sensed measurement of the driver resulting from combining the plurality of driver states; and

modifying control of one or more vehicle systems based on the combined driver state index.

17. The method of claim 16, wherein determining the combined driver state index is based on at least a first driver state selected from the plurality of driver states, a second driver state selected from the plurality of driver states and a third driver state selected from the plurality of driver states, wherein the first driver state, the second driver state and the third driver state are each a different type of driver state.

18. The method of claim 16, wherein determining the combined driver state index is based on at least a first driver state selected from the plurality of driver states, a second driver state selected from the plurality of driver states and a third driver state selected from the plurality of driver states, wherein the first driver state and the second driver state are the same type of driver state and the third driver state is a different type of driver state than the first driver state and the second driver state.

19. The method of claim 17, wherein determining the combined driver state index further includes comparing at least one of the first driver state, the second driver state and the third driver state to respective thresholds, and determining the combined driver state index based on the comparison.

20. The method of claim 19, wherein upon determining the first driver state meets a first driver state threshold and the second driver state meets a second driver state threshold, the combined driver state index is based on the first driver state and the second driver state.

21. The method of claim 17, wherein determining the combined driver state index further includes confirming the first driver state selected from the plurality of driver states with the second driver state selected from the plurality of driver states.

22. The method of claim 21, further including confirming the first driver state selected from the plurality of driver states and the second driver state selected from the plurality of driver states with the third driver state selected from the plurality of driver states.

23. The method of claim 22, wherein confirming the first driver state selected from the plurality of driver states with the second driver state selected from the plurality of driver states further includes comparing the first driver state selected from the plurality of driver states to a first driver state threshold and comparing the second driver state selected from the plurality of driver states to a second driver state threshold.

24. The method of claim 23, wherein upon determining the first driver state selected from the plurality of driver states meets the first driver state threshold and the second driver state selected from the plurality of driver states meets the second driver state threshold, determining the combined driver state index is based on the first driver state selected from the plurality of driver states and the second driver state selected from the plurality of driver states.

25. The method of claim 23, further including determining the first driver state threshold and the second driver state threshold, wherein the first driver state threshold and the second driver state threshold are pre-determined thresholds based on at least one of:

the type of driver state, the monitoring information used to determine the plurality of driver states, and an identity of the driver.

26. The method of claim 25, further including modifying at least one of the first driver state threshold and the second driver state threshold, wherein the first driver state threshold and the second driver state threshold are modified based on the identity of the driver, wherein the identity of the driver is determined by one of the one or more monitoring systems.

27. A method of controlling vehicle systems in a motor vehicle, comprising:

receiving monitoring information from a plurality of monitoring systems;

determining a plurality of driver states associated with a driver based on the monitoring information from the plurality of monitoring systems;

determining a combined driver state index based on the plurality of driver states, wherein the combined driver state index is a variable dependent on a result of combining and confirming each of the plurality of driver states together; and

modifying control of one or more vehicle systems based on the combined driver state index.

28. The method of claim 27, wherein the plurality of driver states being selected from one of the following types of driver states:

physiological driver state, behavioral driver state, and vehicular-sensed driver state.

29. The method of claim 28, further including determining if the driver is distracted or attentive based on at least one of the plurality of driver states and the combined driver state index.

30. The method of claim 29, further including determining a potential hazard based on monitoring information from one or more vehicle systems.

31. The method of claim 30, wherein a first driver state of the plurality of driver states is a behavioral driver state and a second driver state of the plurality of driver states is a behavioral state, and determining if the driver is distracted further includes analyzing the first driver state and the second driver state relative to the potential hazard.

32. The method of claim 30, wherein determining the combined driver state index is based on analyzing at least one of head movement information and hand contact information relative to the potential hazard, wherein the head movement information and the hand contact information are received from the plurality of monitoring systems.

33. The method of claim 32, wherein analyzing head movement information includes determining a head look direction in relation to a direction of the potential hazard and wherein analyzing hand contact information includes determining hand contact with a steering wheel of the vehicle.

34. The method of claim 33, wherein the combined driver state index is based on analyzing the head movement information and upon determining the head look direction is forward-looking relative to the vehicle or the head look direction is directed in the same direction as the direction of the potential hazard, the driver is determined to be attentive, and modifying control of the one or more vehicle systems includes setting a control status of the one or more vehicle systems to no control.

35. The method of claim 33, wherein the combined driver state index is based on analyzing the head movement information and hand contact information and upon determining the head look direction is forward-looking relative to the vehicle or the head look direction is directed in the same direction as the direction of the potential hazard and upon determining at least one hand of the driver is in contact with the steering wheel, the driver is determined to be attentive, and modifying control of the one or more vehicle systems includes setting a control status of the one or more vehicle systems to no control.

36. The method of claim 33, wherein upon determining the potential hazard is a lane deviation based on monitoring information from a lane departure warning system, determining the combined driver state index further includes analyzing head movement information relative to the lane deviation.

37. The method of claim 33, wherein upon determining the potential hazard is a target vehicle in a blind spot monitoring zone of the vehicle based on monitoring information from a blind spot indicator system, determining the combined driver state index further includes analyzing head movement information or hand contact information relative to the target vehicle or the blind spot monitoring zone.

38. The method of claim 33, wherein upon determining the potential hazard is a preceding vehicle in front of the vehicle based on monitoring information from an automatic cruise control system, determining the combined driver state index further includes analyzing head movement information or hand contact information relative to the preceding vehicle.

39. The method of claim 33, wherein upon determining at least one hand is in contact with the steering wheel of the vehicle and a direction of a head look based on the head monitoring information is in a forward-looking direction relative to the vehicle, a control status of a low speed follow system is set to auto control.

40. The method of claim 33, wherein upon determining no hand contact with the steering wheel of the vehicle, a control status of a lane keeping assist system is set to auto control and a control status of an automatic cruise control status is set based on the head monitoring information, wherein the head monitoring information includes the head look direction and a head look duration.

41. The method of claim 33, wherein upon determining at least one hand is in contact with the steering wheel of the vehicle, a control status of an automatic cruise control system is set to manual control and a control status of a lane keeping assist system is set based on the head monitoring information, wherein the head monitoring information includes the head look direction and a head look duration.