Apparatus detecting driving incapability state of driver

• US 10,503,987 B2
• Filed: 06/08/2015
• Issued: 12/10/2019
• Est. Priority Date: 06/23/2014
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Banno, Hiroto, Munaoka, Yutaka, Omi, Takuhiro, Enya, Takeshi
• Primary Examiner(s): Hossain, Farzana
• Application Number:
  US15/320,945
• Publication Number:
  US 20170161575A1
• Time in Patent Office: 1,646 Days
• Field of Search: 348 77, 348148
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1121 Determining geometric values, e.g. centre of rotation or angular range of movement
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2050/0088 Adaptive recalibration
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/10 Accelerator pedal position
• CPC Class Codes: B60W 2540/12 Brake pedal position
• CPC Class Codes: B60W 2540/223 : Posture, e.g. hand, foot, or seat position, turned or inclined
• CPC Class Codes: B60W 2540/225 : Direction of gaze
• CPC Class Codes: B60W 40/02 : related to ambient conditions
• CPC Class Codes: B60W 40/08 : related to drivers or passengers
• CPC Class Codes: B60W 40/10 : related to vehicle motion
• CPC Class Codes: G01S 19/13 : Receivers
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/161 : Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/23 : Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/16 : Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing

Abstract

An apparatus detecting driving incapability state of a driver includes: an inclination detection portion that detects inclination of a head of a driver with respect to a trunk of the driver lower than a neck, based on an image of a driver seat captured by an imaging device mounted on a vehicle; and a posture collapse detection portion that detects the driver is incapable of driving when the inclination of the head detected by the inclination detection portion is greater than a relative inclination threshold during travel of the vehicle.

Claims

1. An apparatus detecting driving incapability state of a driver, the apparatus comprising:

an imaging device that is mounted on a vehicle and is configured to capture a driver'"'"'s seat;

a memory storing an inclination detection portion and a posture collapse detection portion;

a processor executing the inclination detection portion and the posture collapse detection portion stored in the memory;

the inclination detection portion detects inclination of a head of a driver with respect to a trunk of the driver lower than a neck, based on an image of the driver'"'"'s seat captured by the imaging device; and

the posture collapse detection portion detects the driver is incapable of driving when the inclination of the head with respect to the trunk detected by the inclination detection portion is greater than a relative inclination threshold during travel of the vehicle, causing the vehicle to be safely stopped in a case where it is determined that the driver is incapable of driving.

2. The apparatus detecting driving incapability state of the driver according to claim 1, wherein:

the inclination detection portion detects center axes of the head and the trunk from the image, and considers inclination of a center axis of the head with respect to a center axis of the trunk as the inclination of the head.

3. The apparatus detecting driving incapability state of the driver according to claim 1, wherein:

the inclination detection portion detects the head and a seat belt of the driver seat from the image, and detects the inclination of the head from a positional relation between the seat belt and the head.

4. The apparatus detecting driving incapability state of the driver according to claim 1, wherein:

the posture collapse detection portion detects the driver is incapable of driving on condition that a face included in the head does not face in a forward direction of the vehicle for a time longer than a posture collapse determination time.

5. The apparatus detecting driving incapability state of the driver according to claim 1, wherein:

the posture collapse detection portion detects the driver is incapable of driving on condition that the inclination of the head is greater than the relative inclination threshold for a time longer than a posture collapse determination time.

6. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

the posture collapse detection portion detects the driver is incapable of driving on condition that a position of the head and a position of the trunk are within an unmoving determination range for a time longer than the posture collapse determination time.

7. The apparatus detecting driving incapability state of the driver according to claim 5, further comprising:

a learning portion, stored on the memory and executed by the processor, that learns the inclination of the head detected by the inclination detection portion when the driver is not incapable of driving,

wherein;

the posture collapse detection portion detects the driver is incapable of driving on condition that the inclination of the head detected by the inclination detection portion is greater than an inclination determination value by exceeding the inclination of the head learned by the learning portion.

8. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

the posture collapse detection portion detects the driver is incapable of driving on condition that a steering wheel of the vehicle is not operated for a time longer than an operation determination time.

9. The apparatus detecting driving incapability state of the driver according to claim 5, further comprising:

a shake detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when an amplitude of a shake of the head is smaller than a first amplitude or larger than a second amplitude, which is larger than the first amplitude, by a time a shake determination time elapses after external force has been applied to the vehicle, in a case where the posture collapse detection portion does not detect the driver is incapable of driving.

10. The apparatus detecting driving incapability state of the driver according to claim 5, further comprising:

a spasm detection portion, stored on the memory and executed by the processor, that detects spasm of the driver,

wherein;

the relative inclination threshold decreases when the spasm is detected by the spasm detection portion.

11. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

as a vehicle speed of the vehicle is high, the posture collapse determination time is shortened.

12. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

as a time to collision is short, the time being obtained by dividing a vehicle distance from a preceding vehicle by a speed relative to the preceding vehicle, the posture collapse determination time is shortened.

13. The apparatus detecting driving incapability state of the driver according to claim 5, further comprising:

a storage portion in which personal information on the driver, including medical history and age, is registered,

wherein;

the posture collapse determination time is set based on the personal information registered in the storage portion.

14. The apparatus detecting driving incapability state of the driver according to claim 5, wherein:

the posture collapse determination time is prolonged when a drive assist control is executed in the vehicle.

15. The apparatus detecting driving incapability state of the driver according to claim 1, further comprising:

a posture notification portion that notifies the driver of a degree of posture collapse of the driver based on the inclination of the head detected by the inclination detection portion.

16. The apparatus detecting driving incapability state of the driver according to claim 1, wherein:

the driving incapability state includes a state in which the driver has a sudden illness.

17. The apparatus detecting driving incapability state of the driver according to claim 1, further comprising:

a face direction detection portion, stored on the memory and executed by the processor, that detects a direction of the face of the driver with respect to a forward direction of the vehicle based on the image of the driver seat captured by the imaging device mounted on the vehicle; and

a direction collapse detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when the direction of the face detected by the face direction detection portion is greater than a face direction threshold for a time longer than a direction collapse determination time, during travel of the vehicle in a case where the posture collapse detection portion does not detect the driver is incapable of driving.

18. The apparatus detecting driving incapability state of the driver according to claim 1, further comprising:

a face direction detection portion, stored on the memory and executed by the processor, that detects a direction of the face of the driver with respect to a forward direction of the vehicle based on the image of the driver seat captured by the imaging device mounted on the vehicle; and

a direction collapse detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when the direction of the face detected by the face direction detection portion is greater than a face direction threshold and the driver releases a steering wheel of the vehicle, during travel of the vehicle, in a case where the posture collapse detection portion does not detect the driver is incapable of driving.

19. The apparatus detecting driving incapability state of the driver according to claim 1, further comprising:

a face direction detection portion, stored on the memory and executed by the processor, that detects a direction of the face of the driver with respect to a forward direction of the vehicle based on the image of the driver seat captured by the imaging device mounted on the vehicle; and

a direction collapse detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when the direction of the face detected by the face direction detection portion is greater than a face direction threshold and an opening degree of an accelerator is greater than a predetermined opening degree, during travel of the vehicle, in a case where the posture collapse detection portion does not detect the driver is incapable of driving.

20. The apparatus detecting driving incapability state of the driver according to claim 1, further comprising:

a face direction detection portion, stored on the memory and executed by the processor, that detects a direction of the face of the driver with respect to a forward direction of the vehicle based on the image of the driver seat captured by the imaging device mounted on the vehicle; and

a direction collapse detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when the direction of the face detected by the face direction detection portion is greater than a face direction threshold and an accelerator operation and a brake operation are not performed for a time longer than an operation determination time, during travel of the vehicle, in a case where the posture collapse detection portion does not detect the driver is incapable of driving.

21. The apparatus detecting driving incapability state of the driver according to claim 18, wherein:

the direction collapse detection portion detects the driver is incapable of driving on condition that the driver releases the steering wheel for a time longer than an operation determination time.

22. The apparatus detecting driving incapability state of the driver according to claim 19, wherein:

the direction collapse detection portion detects the driver is incapable of driving on condition that the opening degree of the accelerator is larger than the predetermined opening degree for a time longer than an operation determination time.

23. The apparatus detecting driving incapability state of the driver according to claim 19, wherein:

the direction collapse detection portion detects the driver is incapable of driving on condition that the direction of the face is greater than the face direction threshold for a time longer than a direction collapse determination time.

24. The apparatus detecting driving incapability state of the driver according to claim 23, wherein:

the direction collapse detection portion detects the driver is incapable of driving when the face faces downward greatly exceeding a downward threshold or upward greatly exceeding an upward threshold.

25. The apparatus detecting driving incapability state of a driver according to claim 18, wherein:

the direction collapse detection portion detects the driver is incapable of driving on condition that a hand of the driver is lower than a neck of the driver.

26. The apparatus detecting driving incapability state of the driver according to claim 18, further comprising:

a learning portion, stored on the memory and executed by the processor, that learns the direction of the face detected by the face direction detection portion when the driver is not incapable of driving,

wherein;

the direction collapse detection portion detects the driver is incapable of driving on condition that the direction of the face detected by the face direction detection portion is greater than the direction of the face learned by the learning portion by exceeding an inclination determination value.

27. The apparatus detecting driving incapability state of the driver according to claim 1, wherein:

the inclination detection portion detects, on a basis of a neck position of the driver, the inclination of the head with respect to the trunk based on the image of the driver seat; and

the posture collapse detection portion detects the driver is incapable of driving in response to the inclination being greater than a relative inclination threshold.

28. The apparatus detecting driving incapability state of the driver according to claim 1, wherein

the inclination detection portion detects, on a basis of a neck position of the driver, an inclination angle between the head and the trunk based on the image of the driver seat, as the inclination of the head;

and

the posture collapse detection portion determines that the driver is in a driving incapable state in response to a determination that the inclination angle is greater than a predetermined threshold.

29. An apparatus detecting driving incapability state of a driver of a vehicle, the apparatus comprising:

a camera configured to capture a driver'"'"'s seat of the vehicle; and

a processor configured to

detect inclination of a head of the driver with respect to a trunk of the driver lower than a neck of the driver, based on an image of the driver'"'"'s seat captured by the camera, and

detect the driver is incapable of driving when the detected inclination of the head with respect to the trunk is greater than a relative inclination threshold during travel of the vehicle, causing the vehicle to be safely stopped in a case where it is determined that the driver is incapable of driving.

30. A method of detecting driving incapability state of a driver of a vehicle, the method comprising:

capturing, by a camera, a driver'"'"'s seat of the vehicle;

detecting, by a processor, inclination of a head of the driver with respect to a trunk of the driver lower than a neck of the driver, based on an image of the driver'"'"'s seat captured by the camera; and

detecting, by the processor, the driver is incapable of driving when the detected inclination of the head with respect to the trunk is greater than a relative inclination threshold during travel of the vehicle, causing the vehicle to be safely stopped in a case where it is determined that the driver is incapable of driving.