Apparatus detecting driving incapability state of driver

• US 10,474,914 B2
• Filed: 06/08/2015
• Issued: 11/12/2019
• Est. Priority Date: 06/23/2014
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Banno, Hiroto, Omi, Takuhiro, Munaoka, Yutaka, Enya, Takeshi
• Primary Examiner(s): Seth, Manav
• Application Number:
  US15/321,048
• Publication Number:
  US 20170161576A1
• Time in Patent Office: 1,618 Days
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 2540/10 Accelerator pedal position
• CPC Class Codes: B60W 2540/12 Brake pedal position
• CPC Class Codes: B60W 2540/18 Steering angle
• CPC Class Codes: B60W 2540/223 Posture, e.g. hand, foot, or seat position, turned or inclined
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2554/804 Relative longitudinal speed
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/14 Adaptive cruise control
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30201 : Face
• CPC Class Codes: G06T 2207/30241 : Trajectory
• CPC Class Codes: G06T 2207/30268 : Vehicle interior
• CPC Class Codes: G06T 7/292 : Multi-camera tracking
• CPC Class Codes: G06T 7/593 : from stereo images
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 2201/10 : Recognition assisted with metadata
• CPC Class Codes: G06V 2201/12 : Acquisition of 3D measurements of objects
• CPC Class Codes: G06V 40/161 : Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 : Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G08B 21/02 : Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/06 : indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G08G 1/16 : Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

Apparatus detecting driving incapability state of a driver includes: a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver'"'"'s seat captured by an imaging device mounted in a vehicle; and an out-of-frame state detection portion that detects that the driver is incapable of driving, when the head portion detected by the head detection portion is out of a predetermined scope in the image during travelling of the vehicle.

Claims

1. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle;

an out-of-frame state detection portion that detects that the driver is incapable of driving, when the head portion detected by the head detection portion is out of a predetermined scope in the image during travelling of the vehicle; and

a deformation portion that deforms the predetermined scope in accordance with intention of a user including the driver or with driver information on the driver.

3. The apparatus detecting driving incapability state of the driver according to claim 1, wherein:

the driver information includes at least one of age, a medical history, and a habit of driving posture of the driver.

24. The apparatus detecting driving incapability state of the driver according to claim 1, wherein:

a driving incapability state is provided by a state where the driver has a sudden illness.

2. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle;

an out-of-frame state detection portion that detects that the driver is incapable of driving, when the head portion detected by the head detection portion is out of a predetermined scope in the image during travelling of the vehicle; and

a transfer portion that transfers the predetermined scope in accordance with intention of a user including the driver or with driver information on the driver.

25. The apparatus detecting driving incapability state of the driver according to claim 2, wherein:

a driving incapability state is provided by a state where the driver has a sudden illness.

4. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle; and

an out-of-frame state detection portion that detects that the driver is incapable of driving, when the head portion detected by the head detection portion is out of a predetermined scope in the image during travelling of the vehicle, wherein

a shape of the predetermined scope is an ellipse having a major axis extending along a width of the vehicle.

5. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle; and

an out-of-frame state detection portion that detects that the driver is incapable of driving, when the head portion detected by the head detection portion is out of a predetermined scope in the image during travelling of the vehicle, wherein

the out-of-frame state detection portion detects that the driver is incapable of driving, when the head portion is out of the predetermined scope over an out-of-frame determination time.

6. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

a first predetermined scope and a second predetermined scope including the first predetermined scope and a scope outer than the first predetermined scope are at least set as predetermined scopes;

the out-of-frame determination time is set for each of the predetermined scopes; and

the out-of-frame state detection portion detects that the driver is incapable of driving, when the head portion is out of at least one of the predetermined scopes over the out-of-frame determination time corresponding to the predetermined scope.

7. The apparatus detecting driving incapability state of the driver according to claim 6, wherein:

the out-of-frame determination time is set to be shorter as any one of the predetermined scopes is larger.

13. The apparatus detecting driving incapability state of the driver according to claim 5, further comprising:

a quantity detection portion that detects a drawing amount of a seat belt of the driver seat,

wherein;

the out-of-frame state detection portion detects that the driver is incapable of driving, when the drawing amount detected by the quantity detection portion is larger than the drawing amount detected at a time of attachment of the seat belt, by a first drawing amount.

14. The apparatus detecting driving incapability state of the driver according to claim 13, wherein:

the out-of-frame state detection portion detects that the driver is incapable of driving, when the drawing amount per amount detection time, the drawing amount being detected by the quantity detection portion, is larger than a second drawing amount.

15. The apparatus detecting driving incapability state of the driver according to claim 5, further comprising:

a sitting pressure detection portion that detects pressure distribution of a sitting portion of the driver seat,

wherein;

the out-of-frame state detection portion detects that the driver is incapable of driving, when a high-pressure section in the pressure distribution detected by the sitting pressure detection portion is biased to an end of the sitting portion.

16. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

when a position of the head portion successively detected by the head detection portion vibrates with amplitude larger than a predetermined amplitude, the out-of-frame determination time is decreased.

17. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

as a moving speed of the head portion successively detected by the head detection portion is faster, the out-of-frame determination time is decreased.

18. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

when a moving speed of the head portion is faster as a position of the head portion successively detected by the head detection portion approaches an end of the predetermined scope, the out-of-frame determination time is decreased.

19. The apparatus detecting driving incapability state of the driver according to claim 5, further comprising:

a spasm detection portion that detects a spasm of the driver,

wherein;

when the spasm detection portion detects the spasm, the out-of-frame determination time is decreased.

20. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

as a vehicle speed of the vehicle is higher, the out-of-frame determination time is decreased.

21. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

as a time to collision obtained by dividing a distance from a preceding vehicle by a relative speed with respect to the preceding vehicle is smaller, the out-of-frame determination time is decreased.

22. The apparatus detecting driving incapability state of the driver according to claim 5, further comprising:

a storage portion that registers personal information including a medical history and age of the driver,

wherein;

the out-of-frame determination time is set based on the personal information registered in the storage portion.

23. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

when a drive assistance control is performed in the vehicle, the out-of-frame determination time is extended.

26. The apparatus detecting driving incapability state of the driver according to claim 7, wherein:

a driving incapability state is provided by a state where the driver has a sudden illness.

8. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle; and

an out-of-frame state detection portion that detects that the driver is incapable of driving, when the head portion detected by the head detection portion is out of a predetermined scope in the image during travelling of the vehicle, wherein

the imaging device is provided by a stereo camera, and

the predetermined scope is provided by a three-dimensionally set scope.

9. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle;

an out-of-frame state detection portion that detects that the driver is incapable of driving, when the head portion detected by the head detection portion is out of a predetermined scope in the image during travelling of the vehicle; and

a trajectory acquisition portion that acquires a trajectory of the head portion from positions of the head portion detected by the head detection portion,

wherein;

during travelling of the vehicle, the out-of-frame state detection portion detects that the driver is incapable of driving, based on that the head portion detected by the head detection portion is out of a predetermined scope of the image, and the trajectory acquired by the trajectory acquisition portion until the head portion is out of the predetermined scope.

12. The apparatus detecting driving incapability state of the driver according to claim 9, wherein:

when the head detection portion does not detect the head portion, the head detection portion searches a vicinity of a final position of the trajectory acquired by the trajectory acquisition portion.

10. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle;

a trajectory acquisition portion that acquires a trajectory of the head portion from positions of the head portion detected by the head detection portion; and

an out-of-frame state detection portion that detects that the driver is incapable of driving based on the trajectory acquired by the trajectory acquisition portion, during travelling of the vehicle.

11. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a head detection portion that successively detects a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle; and

an out-of-frame state detection portion that detects that the driver is incapable of driving, when the head portion detected by the head detection portion is out of a predetermined scope in the image during travelling of the vehicle, wherein

the predetermined scope is provided by a whole area of the image.

27. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a processor; and

a storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by the processor, perform processing of;

successively detecting a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle;

detecting that the driver is incapable of driving when the head portion is out of a predetermined scope in the image during travelling of the vehicle; and

deforming the predetermined scope in accordance with intention of a user including the driver or with driver information on the driver.

28. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a processor; and

a storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by the processor, perform processing of;

successively detecting a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle;

detecting that the driver is incapable of driving when the head portion is out of a predetermined scope in the image during travelling of the vehicle; and

transferring the predetermined scope in accordance with intention of a user including the driver or with driver information on the driver.

29. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a processor; and

a storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by the processor, perform processing of;

successively detecting a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle; and

detecting that the driver is incapable of driving when the head portion is out of a predetermined scope in the image during travelling of the vehicle, wherein

the driver is detected as being incapable of driving when the head portion is out of the predetermined scope over an out-of-frame determination time.

30. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a processor; and

a storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by the processor, perform processing of;

successively detecting a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle;

detecting that the driver is incapable of driving when the head portion is out of a predetermined scope in the image during travelling of the vehicle; and

acquiring a trajectory of the head portion from positions of the head portion detected,

wherein;

during travelling of the vehicle, the driver is detected as being incapable of driving, based on that the head portion detected is out of a predetermined scope of the image, and the trajectory acquired until the head portion is out of the predetermined scope.

31. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

a processor; and

a storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by the processor, perform processing of;

successively detecting a head portion, which is higher than a neck of the driver, based on an image of a driver seat captured by an imaging device mounted in a vehicle;

acquiring a trajectory of the head portion from positions of the head portion detected; and

detecting that the driver is incapable of driving based on the trajectory acquired, during travelling of the vehicle.