SYSTEM AND METHOD FOR RESPONDING TO DRIVER STATE

• US 20160001781A1
• Filed: 09/11/2015
• Published: 01/07/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Fung, Kin C., Dick, Timothy J.
• Granted Patent: US 9,751,534 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60R 25/25 using biometry
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/45 Detection of the body part being alive
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G07C 5/02 Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00
• CPC Class Codes: G07C 5/08 Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
• CPC Class Codes: G07C 9/00309 operated with bidirectional data transmission between data carrier and locks
• CPC Class Codes: G07C 9/00563 using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A method for controlling vehicle systems includes receiving monitoring information from one or more monitoring systems and determining a plurality of driver states based on the monitoring information from the one or more monitoring systems. The method includes determining a combined driver state based on the plurality of driver states and modifying control of one or more vehicle systems based on the combined driver state.

Claims

1-65. -65. (canceled)

66. A method of controlling vehicle systems in a motor vehicle, comprising:

receiving monitoring information from one or more monitoring systems;

determining a plurality of driver states based on the monitoring information from the one or more monitoring systems;

determining a combined driver state index based on the plurality of driver states; and

modifying control of one or more vehicle systems based on the combined driver state index.

67. The method of claim 66, wherein determining the combined driver state index is based on a first driver state selected from the plurality of driver states and a second driver state selected from the plurality of driver states.

68. The method of claim 67, wherein determining the combined driver state index is further based on a third driver state selected from the plurality of driver states.

69. The method of claim 68, wherein the plurality of driver states being at least one of the following driver state types:

a physiological driver state, a behavioral driver state, or a vehicular-sensed driver state.

70. The method of claim 69, wherein the physiological driver state is based on physiological information, the behavioral driver state is based on behavioral information, and the vehicular-sensed driver state is based on vehicle information.

71. The method of claim 69, wherein the first driver state selected from the plurality of driver states is a physiological driver state, the second driver state selected from the plurality of driver states is a behavioral driver state, and the third driver state selected from the plurality of driver states is a vehicular-sensed driver state.

72. The method of claim 69, wherein the first driver state selected from the plurality of driver states and the second driver state selected from the plurality of driver states are the same driver state type.

73. The method of claim 69, wherein the first driver state is a physiological driver state and the second driver state is a behavioral driver state.

74. The method of claim 73, wherein the monitoring systems include a sensor for receiving physiological information and behavioral information, and the physiological driver state is based on the physiological information and the behavioral driver state is based on the behavioral information received from the sensor.

75. The method of claim 74, wherein the physiological information is heart rate information and the behavioral information is head movement information.

76. The method of claim 74, wherein the sensor is an optical sensor for receiving the physiological information and the behavioral information.

77. The method of claim 68, wherein determining a combined driver state index further includes determining if the combined driver state indicates a distracted driver state or a non-distracted driver state based on the plurality of driver states.

78. The method of claim 77, wherein upon determining at least one of the first driver state selected from the plurality of driver states state or the second driver state selected from the plurality of driver states indicates a distracted driver state, the combined driver state is determined to indicate the distracted driver state.

79. The method of claim 77, wherein upon determining at least one of the first driver state selected from the plurality of driver states or the second driver state selected from the plurality of driver states indicates a non-distracted driver state, the combined driver state is determined to indicate the non-distracted driver state.

80. The method of claim 79, further including upon determining the third driver state selected from the plurality of driver states indicates a distracted driver state, the combined driver state is determined to indicate the distracted driver state.

81. A method of controlling vehicle systems in a motor vehicle, comprising:

receiving monitoring information from one or more monitoring systems;

determining a plurality of driver states based on the monitoring information from the one or more monitoring systems, the plurality of driver states being selected from one of the following types of driver states;

physiological driver state, behavioral driver state, and vehicular-sensed driver state;

determining a combined driver state index based on the plurality of driver states; and

modifying control of one or more vehicle systems based on the combined driver state index.

82. The method of claim 81, wherein determining the combined driver state index is based on at least a first driver state selected from the plurality of driver states, a second driver state selected from the plurality of driver states and a third driver state selected from the plurality of driver states, wherein the first driver state, the second driver state and the third driver state are each a different type of driver state.

83. The method of claim 81, wherein determining the combined driver state index is based on at least a first driver state selected from the plurality of driver states, a second driver state selected from the plurality of driver states and a third driver state selected from the plurality of driver states, wherein the first driver state and the second driver state are the same type of driver state and the third driver state is a different type of driver state than the first driver state and the second driver state.

84. The method of claim 82, wherein determining the combined driver state index further includes comparing at least one of the first driver state, the second driver state and the third driver state to respective thresholds, and determining the combined driver state index based on the comparison.

85. The method of claim 84, wherein upon determining the first driver state meets the first driver state threshold and the second driver state meets the second driver state threshold, the combined driver state index is based on the first driver state and the second driver state.

86. The method of claim 82, wherein determining the combined driver state index further includes confirming the first driver state selected from the plurality of driver states with the second driver state selected from the plurality of driver states.

87. The method of claim 86, further including confirming the first driver state selected from the plurality of driver states and the second driver state selected from the plurality of driver states with the third driver state selected from the plurality of driver states.

88. The method of claim 87, wherein confirming the first driver state selected from the plurality of driver states with the second driver state selected from the plurality of driver states further includes comparing the first driver state selected from the plurality of driver states to a first threshold and comparing the second driver state selected from the plurality of driver states to a second threshold.

89. The method of claim 88, wherein upon determining the first driver state selected from the plurality of driver states meets the first threshold and the second driver state selected from the plurality of driver states meets the second threshold, determining the combined driver state index is based on the first driver state selected from the plurality of driver states and the second driver state selected from the plurality of driver states.

90. The method of claim 88, further including determining the first driver state threshold and the second driver state threshold, wherein the first driver state threshold and the second driver state threshold are pre-determined thresholds based on at least one of:

the type of driver state, the monitoring information used to determine the plurality of driver states, and an identity of the driver.

91. The method of claim 90, further including modifying at least one of the first driver state threshold and the second driver state threshold, wherein the first driver state threshold and the second driver state threshold are modified based on the identity of the driver, wherein the identity of the driver is determined by one of the monitoring systems.

92. A method of controlling vehicle systems in a motor vehicle, comprising:

receiving monitoring information from a plurality of monitoring systems;

determining a plurality of driver states based on the monitoring information from the plurality of monitoring systems;

determining a combined driver state index based on the plurality of driver states; and

modifying control of one or more vehicle systems based on the combined driver state index.

93. The method of claim 92, wherein the plurality of driver states being selected from one of the following types of driver states:

physiological driver state, behavioral driver state, and vehicular-sensed driver state.

94. The method of claim 93, further including determining if the driver is distracted or attentive based on at least one of the plurality of driver states and the combined driver state index.

95. The method of claim 94, further including determining a potential hazard based on monitoring information from one or more vehicle systems.

96. The method of claim 95, wherein a first driver state of the plurality of driver states is a behavioral driver state and a second driver state of the plurality of driver states is a behavioral state, and determining if the driver is distracted further includes analyzing the first driver state and the second driver state relative to the potential hazard.

97. The method of claim 95, wherein determining the combined driver state index is based on analyzing at least one of head movement information and hand contact information relative to the potential hazard, wherein the head movement information and the hand contact information are received from the plurality of monitoring systems.

98. The method of claim 97, wherein analyzing head movement information includes determining a head look direction in relation to a direction of the potential hazard and wherein analyzing hand contact information includes determining hand contact with a steering wheel of the vehicle.

99. The method of claim 98, wherein the combined driver state index is based on analyzing the head movement information and upon determining the head look direction is forward-looking relative to the vehicle or the head look direction is directed in the same direction as the direction of the potential hazard, the driver is determined to be attentive, and modifying control of the one or more vehicle systems includes setting a control status of the one or more vehicle systems to no control.

100. The method of claim 98, wherein the combined driver state index is based on analyzing the head movement information and hand contact information and upon determining the head look direction is forward-looking relative to the vehicle or the head look direction is directed in the same direction as the direction of the potential hazard and upon determining at least one hand of the driver is in contact with the steering wheel, the driver is determined to be attentive, and modifying control of the one or more vehicle systems includes setting a control status of the one or more vehicle systems to no control.

101. The method of claim 98, wherein upon determining the potential hazard is a lane deviation based on monitoring information from a lane departure warning system, determining the combined driver state index further includes analyzing head movement information relative to the lane deviation.

102. The method of claim 98, wherein upon determining the potential hazard is a target vehicle in a blind spot monitoring zone of the vehicle based on monitoring information from a blind spot indicator system, determining the combined driver state index further includes analyzing head movement information or hand contact information relative to the target vehicle or the blind spot monitoring zone.

103. The method of claim 98, wherein upon determining the potential hazard is a preceding vehicle in front of the vehicle based on monitoring information from an automatic cruise control system, determining the combined driver state index further includes analyzing head movement information or hand contact information relative to the preceding vehicle.

104. The method of claim 98, wherein upon determining at least one hand is in contact with the steering wheel of the vehicle and a direction of a head look based on the head monitoring information is in a forward-looking direction relative to the vehicle, a control status of a low speed follow system is set to auto control.

105. The method of claim 98, wherein upon determining no hand contact with the steering wheel of the vehicle, a control status of a lane keeping assist system is set to auto control and a control status of an automatic cruise control status is set based on the head monitoring information, wherein the head monitoring information includes the head look direction and a head look duration.

106. The method of claim 98, wherein upon determining at least one hand is in contact with the steering wheel of the vehicle, a control status of an automatic cruise control system is set to manual control and a control status of a lane keeping assist system is set based on the head monitoring information, wherein the head monitoring information includes the head look direction and a head look duration.