Drowsiness estimation device, drowsiness estimation method, and computer-readable non-transient recording medium

• US 9,501,704 B2
• Filed: 10/04/2013
• Issued: 11/22/2016
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Inventors: Matsuo, Masatoshi
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Yang, Nien-Ru
• Application Number:
  US14/355,078
• Publication Number:
  US 20140313309A1
• Time in Patent Office: 1,145 Days
• Field of Search: 348/78, 348/208.14, 348/576
• US Class Current: 1/1
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A drowsiness estimation device comprises an imaging unit 1, a regional temperature calculation unit 6, and a weighted subtraction unit 7. The imaging unit 1 obtains visible spectrum image data in a visible spectrum capture mode and obtains infra-red image data indicating a surface body temperature distribution for a subject'"'"'s body in an infra-red capture mode. The regional temperature calculation unit 6 detects a temperature of an ocular center region within the surface body temperature distribution indicated by the infra-red image data. The weighted subtraction unit 7 applies a correction to a temperature parameter for drowsiness estimation, based on the detected ocular center region temperature. A drowsiness estimation for the user is then performed according to the corrected parameter.

Claims

1. A drowsiness estimation device detecting a temperature parameter for a drowsiness estimation of a person subject to the drowsiness estimation, and performing the drowsiness estimation, the drowsiness estimation device comprising:

acquisition circuitry acquiring visible spectrum image data obtained by capturing a plurality of images of the person with visible spectrum wavelengths, and surface body temperature distribution data obtained by measuring a temperature distribution of a body surface of the person;

image processing circuitry specifying an ocular region center for the person by performing image processing on the visible spectrum image data; and

correction circuitry detecting a temperature of the ocular region center in the surface body temperature distribution data for the person, and using the temperature of the ocular region center to apply a correction to the temperature parameter for the drowsiness estimation, wherein

the correction to the temperature parameter is performed by multiplying the temperature parameter by a first weighting coefficient and the temperature of the ocular region center by a second weighting coefficient, and subtracting a weighted temperature of the ocular region center from a weighted temperature parameter.

2. The drowsiness estimation device of claim 1, wherein

the image processing circuitry specifies the ocular region center by detecting an outline of a cornea in a face region of the person appearing in the visual spectrum image data, and specifying an arc center as indicated by the outline of the cornea.

3. The drowsiness estimation device of claim 1, wherein

the temperature parameter is a body temperature in a forehead region within a face region of the person appearing in the visual spectrum image data, and

the correction circuitry obtains the temperature parameter by detecting a forehead region temperature from the surface body temperature distribution data.

4. The drowsiness estimation device of claim 1, wherein

the temperature parameter is a body temperature of a portion of a face region of the person appearing in the visual spectrum image data, excluding a mouth region and a hairline region, and

the correction circuitry obtains the temperature parameter by detecting a portion temperature in the face region excluding the mouth region and the hairline region, from the surface body temperature distribution data.

5. The drowsiness estimation device of claim 1, wherein

the surface body temperature distribution data is infra-red image data made up of a plurality of pixels and having a predetermined resolution,

the pixels of the infra-red image data are in correspondence with pixels of the visible spectrum image data, and

a color component luminance of each of the pixels of the infra-red image data indicates an amount of infra-red radiation emitted by a corresponding portion of the body surface of the person appearing in the visual spectrum image data.

6. The drowsiness estimation device of claim 5, wherein

the visual spectrum image data and the infra-red image data differ in resolution,

the image processing circuitry specifies the ocular region center using an X-coordinate or a Y-coordinate in a coordinate system of the visual spectrum image data,

the correction circuitry applies a conversion to the X-coordinate or the Y-coordinate of the ocular region center and converts a pixel value from the infra-red image data at a converted X-coordinate or a converted Y-coordinate into a temperature, and

the conversion applied by the correction circuitry involves multiplying the X-coordinate or the Y-coordinate of the ocular region center by a horizontal pixel ratio or a vertical pixel ratio of the visible spectrum image data and the infra-red image data, and then adding a horizontal offset or a vertical offset representing a cause of difference between a visible spectrum imaging system and an infra-red spectrum imaging system.

7. The drowsiness estimation device of claim 6, wherein

the visible spectrum image data and the infra-red image data are obtained by capturing the images of the person at a plurality of imaging times in a measurement time slot,

at each of the imaging times, the image processing circuitry specifies the ocular region center, the acquisition circuitry acquires the infra-red image data, and the correction circuitry applies the correction to the temperature parameter, and

the drowsiness estimation is performed by comparing a corrected temperature parameter for a given measurement time to a past corrected temperature parameter for a past measurement time, and determining whether a downward trend is occurring and whether a span of the downward trend exceeds a predetermined threshold.

8. The drowsiness estimation device of claim 5, further comprising

imaging circuitry that is switchable to one of a first mode of being transmissive to the visible spectrum wavelengths and opaque to infra-red wavelengths, and a second mode of being transmissive to the infra-red wavelengths and opaque to the visible spectrum wavelengths, wherein

the visible spectrum image data and the infra-red image data are respectively obtained by switching between the first mode and the second mode.

9. The drowsiness estimation device of claim 1, wherein

the temperature parameter for the drowsiness estimation is acquired by a contact sensor attached to an arm, a leg, and a collarbone of the person.

10. A drowsiness estimation method used by a drowsiness estimation device detecting a temperature parameter for a drowsiness estimation of a person subject to the drowsiness estimation, and performing the drowsiness estimation, the drowsiness estimation method comprising:

acquiring visible spectrum image data obtained by capturing a plurality of images of the person with visible spectrum wavelengths, and surface body temperature distribution data obtained by measuring a temperature distribution of a body surface of the person;

performing image processing on the visible spectrum image data by specifying an ocular region center for the person; and

applying a correction to the temperature parameter for the drowsiness estimation by detecting a temperature of the ocular region center in the surface body temperature distribution data for the person, and using the temperature of the ocular region center to apply the correction, wherein

the correction to the temperature parameter is performed by multiplying the temperature parameter by a first weighting coefficient and the temperature of the ocular region center by a second weighting coefficient, and subtracting a weighted temperature of the ocular region center from a weighted temperature parameter.

11. A computer-readable non-transitory recording medium on which one or more program codes are recorded for causing a computer to perform a drowsiness estimation of a person subject to the drowsiness estimation using a temperature parameter, by causing the computer to execute processing of:

acquiring visible spectrum image data obtained by capturing a plurality of images of the person with visible spectrum wavelengths, and surface body temperature distribution data obtained by measuring a temperature distribution of a body surface of the person;

performing image processing on the visible spectrum image data by specifying an ocular region center for the person; and

applying a correction to the temperature parameter for the drowsiness estimation by detecting a temperature of the ocular region center in the surface body temperature distribution data for the person, and using the temperature of the ocular region center to apply the correction, wherein

the correction to the temperature parameter is performed by multiplying the temperature parameter by a first weighting coefficient and the temperature of the ocular region center by a second weighting coefficient, and subtracting a weighted temperature of the ocular region center from a weighted temperature parameter.