DRIVER MONITORING APPARATUS, DRIVER MONITORING METHOD, AND PROGRAM

• US 20190071095A1
• Filed: 08/01/2018
• Published: 03/07/2019
• Est. Priority Date: 09/01/2017
• Status: Active Grant

• Current Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.), Toshiba Digital Solutions Corporation (Toshiba Corporation)
• Original Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Inventors: Hashimoto, Shinya
• Granted Patent: US 10,457,293 B2
• Time in Patent Office: Days
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A driver monitoring apparatus has a photographing means for taking a picture of a driver'"'"'s face, an illumination means for emitting light to the driver'"'"'s face, a light amount control means for setting the amount of light from the illumination means to a first or second amount of light, a differential image creating means for comparing image information about one image on which the driver'"'"'s face to which the first amount of light was emitted is photographed with image information about another image on which the driver'"'"'s face to which the second amount of light was emitted is photographed and then creating a differential image from differential information about brightness of corresponding pixels, a detecting means for detecting the driver'"'"'s eyes from the differential image created by the differential image creating means, and a state detecting means for detecting the driver'"'"' state according to an image of the driver'"'"'s eyes.

Claims

1. A driver monitoring apparatus comprising:

a photographing means for taking a picture of the face of a driver;

an illumination means for emitting light to the face of the driver;

a light amount control means for controlling an amount of light to be emitted from the illumination means so that the amount of light becomes a first amount of light or a second amount of light;

a differential image creating means for comparing image information about one image on which the face of the driver to which the first amount of light was emitted is photographed with image information about another image on which the face of the driver to which the second amount of light was emitted is photographed and then creating a differential image from differential information about brightness of corresponding pixels;

a detecting means for detecting the eyes of the driver from the differential image created by the differential image creating means; and

a state detecting means for detecting a state of the driver according to an image of the eyes of the driver.

2. The driver monitoring apparatus according to claim 1, wherein one of the first amount of light and the second amount of light is zero.

3. The driver monitoring apparatus according to claim 1, further comprising a correcting means for correcting brightness in the differential image created by the differential image creating means, wherein

the detecting means detects the eyes of the driver from a differential image resulting from correction by the correcting means.

4. The driver monitoring apparatus according to claim 1, wherein the light amount control means controls the amount of light to be emitted from the illumination means so that the amount of light becomes the first amount of light or the second amount of light in synchronization with a photographing timing taken by the photographing means.

5. The driver monitoring apparatus according to claim 1, further comprising a brightness correcting means for creating pre-automatic-brightness-correction image information by subtracting an amount of correction for brightness that has been automatically corrected by the photographing means from brightness in image information obtained from the photographing means, wherein

the differential image creating means creates differential image information from the pre-automatic-brightness-correction image information.

6. The driver monitoring apparatus according to claim 1, further comprising an image switching means for making a switchover to select image information to be supplied to the detecting means depending on illumination on an image of the face of the driver, wherein

if the image switching means decides that the illumination on the image of the face of the driver is lower than a predetermined threshold illumination, the image switching means supplies, to the detecting means, image information based on the differential image created by the differential image creating means,

if the image switching means decides that the illumination on the image of the face of the driver is higher than the predetermined threshold illumination, the image switching means supplies, to the detecting means, the image information obtained from the photographing means, and

the detecting means detects the eyes of the driver according to the supplied image information.

7. The driver monitoring apparatus according to claim 1, wherein the state detecting means detects an orientation of the face of the driver.

8. The driver monitoring apparatus according to claim 1, wherein the state detecting means detects an open/closed state of the eyes of the driver.

9. A driver monitoring method comprising:

a photographing step of taking a plurality of pictures of the face of a driver with different amounts of light emitted to the face of the driver;

a differential image creating step of comparing image information about one image on which the face of the driver to which a first amount of light was emitted is photographed with image information about another image on which the face of the driver to which a second amount of light was emitted is photographed and then creating a differential image from differential information about brightness of corresponding pixels;

a detecting step of detecting the eyes of the driver from the differential image; and

a state detecting step of detecting a state of the driver according to an image of the eyes of the driver.

10. The driver monitoring method according to claim 9, wherein one of the first amount of light and the second amount of light is zero.

11. The driver monitoring method according to claim 9, further comprising a correcting step for correcting brightness in the differential image created by the differential image creating step, wherein

the detecting step detects the eyes of the driver from a differential image resulting from correction by the correcting step.

12. The driver monitoring method according to claim 9, wherein the amount of light to be emitted becomes the first amount of light or the second amount of light in synchronization with a photographing timing in the photographing step.

13. The driver monitoring method according to claim 9, further comprising a brightness correcting step for creating pre-automatic-brightness-correction image information by subtracting an amount of correction for brightness that has been automatically corrected by a photographing means from brightness in image information obtained from the photographing means, wherein

the differential image creating step creates differential image information from the pre-automatic-brightness-correction image information.

14. The driver monitoring method according to claim 9, further comprising an image switching step for making a switchover to select image information to be supplied to the detecting step depending on illumination on an image of the face of the driver, wherein

if the image switching step decides that the illumination on the image of the face of the driver is lower than a predetermined threshold illumination, the image switching step supplies, to the detecting step, image information based on the differential image created by the differential image creating step,

if the image switching step decides that the illumination on the image of the face of the driver is higher than the predetermined threshold illumination, the image switching step supplies, to the detecting step, the image information obtained from the photographing step, and

the detecting step detects the eyes of the driver according to the supplied image information.

15. The driver monitoring method according to claim 9, wherein the state detecting step detects an orientation of the face of the driver.

16. The driver monitoring method according to claim 9, wherein the state detecting step detects an open/closed state of the eyes of the driver.

17. A program that causes a computer to function as:

a photographing means for taking a picture of the face of a driver;

an illumination means for emitting light to the face of the driver;

a light amount control means for controlling an amount of light to be emitted from the illumination means so that the amount of light becomes a first amount of light or a second amount of light;

a differential image creating means for comparing image information about one image on which the face of the driver to which the first amount of light was emitted is photographed with image information about another image on which the face of the driver to which the second amount of light was emitted is photographed and then creating a differential image from differential information about brightness of corresponding pixels;

a detecting means for detecting the eyes of the driver from the differential image created by the differential image creating means; and

a state detecting means for detecting a state of the driver according to an image of the eyes of the driver.