Driving consciousness estimation device

• US 10,640,122 B2
• Filed: 04/05/2017
• Issued: 05/05/2020
• Est. Priority Date: 04/28/2016
• Status: Active Grant

• Current Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Kishi, Hiroshi, Akutsu, Masakazu, Akamatsu, Motoyuki, Sato, Toshihisa
• Primary Examiner(s): Badii, Behrang
• Assistant Examiner(s): Greene, Daniel L
• Application Number:
  US15/479,539
• Publication Number:
  US 20170313319A1
• Time in Patent Office: 1,126 Days
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/333 Lasers
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 2360/77 Instrument locations other than the dashboard
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60N 2002/981 Warning systems, e.g. the seat or seat parts vibrates to warn the passenger when facing a danger
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 : including control of braking systems
• CPC Class Codes: B60W 10/20 : including control of steering systems
• CPC Class Codes: B60W 2040/0818 : Inactivity or incapacity of driver
• CPC Class Codes: B60W 2050/143 : Alarm means
• CPC Class Codes: B60W 2050/146 : Display means
• CPC Class Codes: B60W 2420/403 : Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 : Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2520/105 : Longitudinal acceleration
• CPC Class Codes: B60W 2520/125 : Lateral acceleration
• CPC Class Codes: B60W 2520/14 : Yaw
• CPC Class Codes: B60W 2520/28 : Wheel speed
• CPC Class Codes: B60W 2540/221 : Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 2540/223 : Posture, e.g. hand, foot, or seat position, turned or inclined
• CPC Class Codes: B60W 2540/225 : Direction of gaze
• CPC Class Codes: B60W 2540/229 : Attention level, e.g. attentive to driving, reading or sleeping
• CPC Class Codes: B60W 2540/26 : Incapacity
• CPC Class Codes: B60W 2552/20 : Road profile, i.e. the change in elevation or curvature of a plurality of continuous road segments
• CPC Class Codes: B60W 2552/30 : Road curve radius
• CPC Class Codes: B60W 2554/80 : Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/801 : Lateral distance
• CPC Class Codes: B60W 2554/802 : Longitudinal distance
• CPC Class Codes: B60W 2554/804 : Relative longitudinal speed
• CPC Class Codes: B60W 2556/50 : of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2720/106 : Longitudinal acceleration
• CPC Class Codes: B60W 2720/125 : Lateral acceleration
• CPC Class Codes: B60W 2720/14 : Yaw
• CPC Class Codes: B60W 2720/28 : Wheel speed
• CPC Class Codes: B60W 2754/30 : Longitudinal distance
• CPC Class Codes: B60W 2754/50 : Relative longitudinal speed
• CPC Class Codes: B60W 30/08 : Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 40/08 : related to drivers or passengers
• CPC Class Codes: B60W 50/082 : Selecting or switching between different modes of propelling
• CPC Class Codes: B60W 50/12 : Limiting control by the driver depending on vehicle state, e.g. interlocking means for the control input for preventing unsafe operation
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 50/16 : Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: B60W 60/0053 : from vehicle to occupant
• CPC Class Codes: B60W 60/0059 : Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity
• CPC Class Codes: G01S 19/13 : Receivers
• CPC Class Codes: G05D 1/0248 : in combination with a laser lasers per se H01S
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A driving consciousness estimation device includes a driving readiness estimation unit configured to estimate a driving readiness relating to a driving consciousness of the driver from a driver'"'"'s reaction to the travelling environment, a driving task demand estimation unit configured to estimate a driving task demand which is an index required for the driver with respect to the driving readiness from the travelling environment, and an attention awakening unit configured to execute awakening of attention for the driver relating to the driving of the vehicle based on the result of comparison between the driving readiness and the driving task demand.

Claims

1. A driving consciousness estimation device comprising:

a driver image acquisition unit configured to acquire a driver image in which a driver of a vehicle is imaged;

a travelling environment recognition unit configured to recognize a travelling environment around the vehicle;

a driving readiness estimation unit configured to estimate a driving readiness relating to a driving consciousness of the driver from a driver'"'"'s reaction to the travelling environment based on the driver image and the travelling environment;

a driving task demand estimation unit configured to estimate a driving task demand;

a margin calculation unit configured to calculate a result of comparison between a value of the driving readiness and a value of the driving task demand as a margin; and

an attention awakening unit configured to execute awakening of attention for the driver relating to the driving of the vehicle,

wherein the vehicle is capable of autonomous driving by shifting between a plurality of autonomous driving levels, the levels of autonomous driving being defined according to a degree of autonomous driving,

wherein the driving readiness is a degree of height of the driving consciousness of the driver,

wherein the driving task demand is calculated based on at least one of the following;

a vehicle speed of the vehicle,

time to collision between the vehicle and a preceding vehicle, and

a curvature of a road that the vehicle is traveling,

wherein the driving task demand is a required driving readiness required from the traveling environment, wherein, in a case where the margin is smaller than a margin threshold value, the attention awakening unit is configured to execute awakening of the attention, and

wherein the higher the executed autonomous driving level is in the vehicle, the smaller the margin threshold value used.

2. The driving consciousness estimation device according to claim 1, further comprising:

a manual driving possibility determination unit configured to determine whether or not the driver is in a manual driving possible situation based on the result of comparison between the driving readiness and the driving task demand when an autonomous driving electronic control unit (ECU) of the vehicle ends the autonomous driving.

3. The driving consciousness estimation device according to claim 1,

wherein the driving readiness estimation unit is configured to calculate the number of saccades of eye balls per a predetermined time interval during a period when the driver is facing the extending direction of the road on which the vehicle is travelling based on the driver image and the travelling environment, and estimate the driving readiness based on the number of saccades.

4. The driving consciousness estimation device according to claim 3, further comprising:

a manual driving possibility determination unit configured to determine whether or not the driver is in a manual driving possible situation based on the result of comparison between the driving readiness and the driving task demand when an autonomous driving electronic control unit (ECU) of the vehicle ends the autonomous driving.

5. A driving consciousness estimation device comprising:

a driver image acquisition unit configured to acquire a driver image in which a driver of a vehicle is imaged;

a travelling environment recognition unit configured to recognize a travelling environment around the vehicle;

a vehicle state recognition unit configured to recognize a travelling state of the vehicle;

a driving readiness estimation unit configured to estimate a driving readiness relating to a driving consciousness of the driver from a driver'"'"'s reaction to the vehicle state based on the driver image and the vehicle state;

a driving task demand estimation unit configured to estimate a driving task demand;

a margin calculation unit configured to calculate a result of comparison between a value of the driving readiness and a value of the driving task demand as a margin; and

an attention awakening unit configured to execute awakening of attention for the driver relating to the driving of the vehicle,

wherein the vehicle is capable of autonomous driving by shifting between a plurality of autonomous driving levels, the levels of autonomous driving being defined according to a degree of autonomous driving,

wherein the driving readiness is a degree of height of the driving consciousness of the driver,

wherein the driving task demand is calculated based on at least one of the following;

a vehicle speed of the vehicle,

time to collision between the vehicle and a preceding vehicle, and

a curvature of a road that the vehicle is traveling,

wherein the driving task demand is a required driving readiness required from the traveling environment,

wherein, in a case where the margin is smaller than a margin threshold value, the attention awakening unit is configured to execute awakening of the attention, and

wherein the higher the executed autonomous driving level is in the vehicle, the smaller the margin threshold value used.

6. The driving consciousness estimation device according to claim 5, further comprising:

a manual driving possibility determination unit configured to determine whether or not the driver is in a manual driving possible situation based on the result of comparison between the driving readiness and the driving task demand when an autonomous driving electronic control unit (ECU) of the vehicle ends the autonomous driving.

7. The driving consciousness estimation device according to claim 5,

wherein the vehicle state recognition unit is configured to recognize a longitudinal acceleration of the vehicle as the vehicle state, and

wherein the driving readiness estimation unit is configured to estimate the driving readiness from a head pitch angle of the driver with respect to the longitudinal acceleration of the vehicle based on the driver image and the longitudinal acceleration of the vehicle.

8. The driving consciousness estimation device according to claim 7, further comprising:

a manual driving possibility determination unit configured to determine whether or not the driver is in a manual driving possible situation based on the result of comparison between the driving readiness and the driving task demand when an autonomous driving electronic control unit (ECU) of the vehicle ends the autonomous driving.