Bodily state detection apparatus

• US 5,795,306 A
• Filed: 08/07/1996
• Issued: 08/18/1998
• Est. Priority Date: 03/10/1994
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Shimotani, Mitsuo, Okawa, Futoshi, Okada, Akira, Satake, Toshihide, Terashita, Hiromi, Washino, Shoichi, Nishida, Minoru
• Primary Examiner(s): HINDENBURG, MAX F
• Application Number:
  US08/692,236
• Time in Patent Office: 741 Days
• Field of Search: 128/745, 340/573, 340/575, 340/576
• US Class Current: 600/558
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/6821 Eye
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A bodily state detection apparatus comprising a CCD camera, an infrared LED device, a pickup image memory, a pupil extraction circuit and a bodily state judgment circuit. The CCD camera inputs images of a predetermined area including the subject person'"'"'s face. The infrared LED device illuminates the subject person in such a way that the optical axis of the camera and the direction of the illumination coincide with each other. The pickup image memory stores temporarily the output data of the CCD camera. The pupil extraction circuit extracts the subject person'"'"'s pupil position from the pickup images. The bodily state judgment circuit judges the subject person'"'"'s bodily state by use of the result of pupil extraction performed by the pupil extraction circuit.

Claims

1. A bodily state detection apparatus comprising:

optical input means for receiving images of a predetermined area including the face of a subject person;

illumination means for illuminating said subject person, said illumination means being positioned so that at least near said subject person, the direction of the illumination coincides substantially with the optical axis connecting said subject person with said optical input means;

pupil extraction means for extracting either the pupil position or the pupil shape of said subject person from the images thereof output from said optical input means;

search range definition means for defining the range of search for feature points in said subject person'"'"'s face on the basis of the result of the pupil extraction performed by said pupil extraction means;

feature quantity extraction means for extracting either the position or the shape of said feature points in the range defined by said search range definition means; and

bodily state judgment means for judging the bodily state of said subject person in accordance with the result of the feature quantity extraction performed by said feature quantity extraction means.

2. A bodily state detection apparatus according to claim 1, further comprising extraction result storage means for storing the result of said feature quantity extraction, wherein said bodily state judgment means judges the bodily state of said subject person in accordance with the stored contents of said extraction result storage means.

3. A bodily state detection apparatus according to claim 1, further comprising eyelid opening calculation means for calculating the degree of eyelid opening of said subject person in accordance with the result of eyelid feature quantity extraction performed by said feature quantity extraction means, wherein said bodily state judgment means judges the bodily state of said subject person on the basis of the calculated degree of eyelid opening.

4. A bodily state detection apparatus according to claim 1, further comprising gaze direction detection means for detecting the gaze direction of said subject person in accordance with the result of pupil extraction by said pupil extraction means and with the result of feature quantity extraction by said feature quantity extraction means, wherein said bodily state judgment means judges the bodily state of said subject person on the basis of time series patterns of said subject person'"'"'s gaze direction.

5. A bodily state detection apparatus according to claim 1, further comprising stimulus generation means for generating stimulus to said subject person depending on the primary result of judgment by said bodily state judgment means, wherein said bodily state judgment means makes a secondary judgment on the bodily state of said subject person in accordance with the status of the feature points before and after the generation of said stimulus.

6. A bodily state detection apparatus according to claim 1, further comprising face distance measurement means for measuring the distance between a predetermined position and the face of said subject person, wherein said feature quantity extraction means corrects the extracted feature quantity in accordance with the measured distance.

7. A bodily state detection apparatus according to claim 1, further comprising face orientation estimation means for measuring the face orientation of said subject person, wherein said feature quantity extraction means corrects the extracted feature quantity in accordance with the measured face orientation.