Bodily state detection apparatus

• US 5,573,006 A
• Filed: 10/27/1994
• Issued: 11/12/1996
• Est. Priority Date: 03/10/1994
• Status: Expired due to Term

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Shimotani, Mitsuo, Okawa, Futoshi, Okada, Akira, Satake, Toshihide, Nishida, Minoru, Terashita, Hiromi, Washino, Shoichi
• Primary Examiner(s): HINDENBURG, MAX F
• Application Number:
  US08/330,322
• Time in Patent Office: 747 Days
• Field of Search: 128/745, 340/573, 340/575, 340/576
• US Class Current: 600/558
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/6821 Eye
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A bodily state detection apparatus comprising a CCD camera, an infrared LED device, a pickup image memory, a pupil extraction circuit and a bodily state judgment circuit. The CCD camera inputs images of a predetermined area including the subject person'"'"'s face. The infrared LED device illuminates the subject person in such a way that the optical axis of the camera and the direction of the illumination coincide with each other. The pickup image memory stores temporarily the output data of the CCD camera. The pupil extraction circuit extracts the subject person'"'"'s pupil position from the pickup images. The bodily state judgment circuit judges the subject person'"'"'s bodily state by use of the result of pupil extraction performed by the pupil extraction circuit.

Claims

1. A bodily state detection apparatus comprising:

optical input means for receiving images of a predetermined area including the face of a subject person;

illumination means for illuminating said subject person, said illumination means being positioned so that at least near said subject person, the direction of illumination substantially coincides with an optical axis from said subject person to said optical input means;

pupil extraction means for extracting at least one of pupil position and pupil shape of said subject person based upon brightest locations of the images output from said optical input means; and

bodily state judgment means for judging the bodily state of said subject person by use of the result of the pupil extraction performed by said pupil extraction means.

2. A bodily state detection apparatus according to claim 1, further comprising extraction result storage means for storing the result of said pupil extraction, wherein said bodily state judgment means judges the bodily state of said subject person by use of the stored contents of said extraction result storage means.

3. A bodily state detection apparatus according to claim 1, further comprising eyelid opening estimation means for estimating the degree of eyelid opening of said subject person in accordance with the pupil shape extracted by said pupil extraction means, wherein said bodily state judgment means judges the bodily state of said subject person by use of the estimated degree of eyelid opening.

4. A bodily state detection apparatus according to claim 1, further comprising blinking frequency estimation means for estimating the frequency of blinking of said subject person in accordance with the result of said pupil extraction, wherein said bodily state judgment means judges the bodily state of said subject person on the basis of said frequency of blinking.

5. A bodily state detection apparatus according to claim 1, further comprising pupil waveform output means for measuring the diameter of the pupils from the pupil shape extracted by said pupil extraction means and for outputting waveforms of expansion and contraction of the pupils of said subject person, wherein said bodily state judgment means judges the bodily state of said subject person in accordance with said waveforms of expansion and extraction of the pupils.

6. A bodily state detection apparatus according to claim 5, wherein said pupil waveform output means measures the maximum diameter of the pupils from the pupil shape extracted by said pupil extraction means, thereby outputting the waveforms of expansion and contraction of the pupils of said subject person.

7. A bodily state detection apparatus according to claim 1, further comprising stimulus generation means for generating stimulus to said subject person depending on the primary result of judgment by said bodily state judgment means, wherein said bodily state judgment means makes a secondary judgment on the bodily state of said subject person in accordance with the status of the pupils before and after the generation of said stimulus.

8. A bodily state detection apparatus according to claim 1, further comprising face distance measurement means for measuring the distance between a predetermined position and the face of said subject person, wherein said pupil extraction means corrects the pupil shape in accordance with the measured distance.

9. A bodily state detection apparatus according to claim 1, further comprising illumination measurement means for measuring the intensity of illumination near said subject person, wherein said bodily state judgment means judges the bodily state of said subject person in accordance with the result of the measured intensity of illumination.

10. A bodily state detection apparatus according to claim 1, further comprising face orientation estimation means for measuring the face orientation of said subject person, wherein said pupil extraction means corrects the pupil shape in accordance with the measured face orientation.

11. A bodily state detection apparatus according to claim 1, further comprising magnification means for magnifying the images of said predetermined area including the face of said subject person, the magnified images being input to said optical input means.

12. The bodily state detection apparatus of claim 1, wherein said illumination means includes:

a light generator positioned away from the optical axis from said subject person to said optical input means; and

a light reflector which reflects light from the light generator toward said subject person along the optical axis from said subject person and said optical input means.

13. The bodily state detection apparatus of claim 12, wherein said light reflector is positioned along the optical axis from said subject person to said optical input means, and allows light reflected from said subject person to pass to said optical input means.

14. The bodily state detection apparatus of claim 13, wherein said light reflector is a half mirror.

15. The bodily state detection apparatus of claim 12, wherein said light generator is an infrared LED.