DROWSINESS ESTIMATION DEVICE, DROWSINESS ESTIMATION METHOD, AND COMPUTER-READABLE NON-TRANSIENT RECORDING MEDIUM

• US 20140313309A1
• Filed: 10/04/2013
• Published: 10/23/2014
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Matsuo, Masatoshi
• Granted Patent: US 9,501,704 B2
• Time in Patent Office: Days
• US Class Current: 348/78
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A drowsiness estimation device comprises an imaging unit 1, a regional temperature calculation unit 6, and a weighted subtraction unit 7. The imaging unit 1 obtains visible spectrum image data in a visible spectrum capture mode and obtains infra-red image data indicating a surface body temperature distribution for a subject'"'"'s body in an infra-red capture mode. The regional temperature calculation unit 6 detects a temperature of an ocular centre region within the surface body temperature distribution indicated by the infra-red image data. The weighted subtraction unit 7 applies a correction to a temperature parameter for drowsiness estimation, based on the detected ocular centre region temperature. A drowsiness estimation for the user is then performed according to the corrected parameter.

Claims

1. A drowsiness estimation device detecting a temperature parameter for a drowsiness estimation of a person subject to the drowsiness estimation, and performing the drowsiness estimation, the drowsiness estimation device comprising:

an acquisition unit acquiring visible spectrum image data obtained by capturing a plurality of images of the person with visible spectrum wavelengths, and surface body temperature distribution data obtained by measuring a temperature distribution of a body surface of the person;

an image processing unit specifying an ocular region centre for the person by performing image processing on the visible spectrum image data; and

a correction unit detecting a temperature of the ocular region centre in the surface body temperature distribution data for the person, and using the temperature of the ocular region centre to apply a correction to the temperature parameter for the drowsiness estimation.

2. The drowsiness estimation device of claim 1, wherein

the image processing unit specifies the ocular region centre by detecting an outline of a cornea in a face region of the person appearing in the visual spectrum image data, and specifying an arc centre as indicated by the outline of the cornea.

3. The drowsiness estimation device of claim 1, wherein

the temperature parameter is a body temperature in a forehead region within a face region, and

the correction unit obtains the temperature parameter by detecting a forehead region temperature from the surface body temperature distribution data.

4. The drowsiness estimation device of claim 1, wherein

the temperature parameter is a body temperature of a portion of a face region, excluding a mouth region and a hairline region, and

the correction unit obtains the temperature parameter by detecting a portion temperature in the face region excluding the mouth region and the hairline region, from the surface body temperature distribution data.

5. The drowsiness estimation device of claim 1, wherein

the correction is applied to the temperature parameter by multiplying the temperature parameter and the temperature of the ocular region centre by respective first and second weighting coefficients, and subtracting a weighted temperature of the ocular region centre from a weighted temperature parameter.

6. The drowsiness estimation device of claim 1, wherein

the surface body temperature distribution data are infra-red image data made up of a plurality of pixels and having a predetermined resolution,

the pixels of the infra-red image data are in correspondence with pixels of the visible spectrum image data, and

a colour component luminance of each of the pixels of the infra-red image data indicates an amount of infra-red radiation emitted by a corresponding portion of the body surface of the person appearing in the visual spectrum image data.

7. The drowsiness estimation device of claim 6, wherein

the visual spectrum image data and the infra-red image data differ in resolution,

the image processing unit specifies the ocular region centre using an X-coordinate or a Y-coordinate in a coordinate system of the visual spectrum image data,

the correction unit applies a conversion to the X-coordinate or the Y-coordinate of the ocular region centre and converts a pixel value from the infra-red image data at a converted X-coordinate or a converted Y-coordinate into a temperature, and

the conversion applied by the correction unit involves multiplying the X-coordinate or the Y-coordinate of the ocular region centre by a horizontal pixel ratio or a vertical pixel ratio of the visible spectrum image data and the infra-red image data, and then adding a horizontal offset or a vertical offset representing a cause of difference between a visible spectrum imaging system and an infra-red spectrum imaging system.

8. The drowsiness estimation device of claim 7, wherein

the visible spectrum image data and the infra-red image data are obtained by capturing the images of the person at a plurality of imaging times in a measurement time slot,

at each of the imaging times, the image processing unit specifies the ocular region centre, the acquisition unit acquires the infra-red image data, and the correction unit applies the correction to the temperature parameter, and

the drowsiness estimation is performed by comparing a corrected temperature parameter for a given measurement time to a past corrected temperature parameter for a past measurement time, and determining whether a downward trend is occurring and whether a span of the downward trend exceeds a predetermined threshold.

9. The drowsiness estimation device of claim 6, further comprising

an imaging unit that is switchable to one of a first mode of being transmissive to the visible spectrum wavelengths and opaque to infra-red wavelengths, and a second mode of being transmissive to the infra-red wavelengths and opaque to the visible spectrum wavelengths, wherein

the visible spectrum image data and the infra-red image data are respectively obtained by switching between the first mode and the second mode.

10. The drowsiness estimation device of claim 1, wherein

the temperature parameter for the drowsiness estimation is acquired by a contact sensor attached to an arm, a leg, and a collarbone of the person.

11. A drowsiness estimation method used by a drowsiness estimation device detecting a temperature parameter for a drowsiness estimation of a person subject to the drowsiness estimation, and performing the drowsiness estimation, the drowsiness estimation method comprising:

acquiring visible spectrum image data obtained by capturing a plurality of images of the person with visible spectrum wavelengths, and surface body temperature distribution data obtained by measuring a temperature distribution of a body surface of the person;

performing image processing on the visible spectrum image data by specifying an ocular region centre for the person; and

applying a correction to the temperature parameter for the drowsiness estimation by detecting a temperature of the ocular region centre in the surface body temperature distribution data for the person, and using the temperature of the ocular region centre to apply the correction.

12. A computer-readable non-transitory recording medium on which one or more program codes are recorded for causing a computer to perform a drowsiness estimation of a person subject to the drowsiness estimation using a temperature parameter, by causing the computer to execute processing of:

acquiring visible spectrum image data obtained by capturing a plurality of images of the person with visible spectrum wavelengths, and surface body temperature distribution data obtained by measuring a temperature distribution of a body surface of the person;

performing image processing on the visible spectrum image data by specifying an ocular region centre for the person; and

applying a correction to the temperature parameter for the drowsiness estimation by detecting a temperature of the ocular region centre in the surface body temperature distribution data for the person, and using the temperature of the ocular region centre to apply the correction.