Eye tracking image pickup apparatus for separating noise from feature portions

• US 5,016,282 A
• Filed: 07/12/1989
• Issued: 05/14/1991
• Est. Priority Date: 07/14/1988
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: ATR Communication Systems Research Laboratories
• Inventors: Tomono, Akira, Iida, Muneo, Ohmura, Kazunori
• Primary Examiner(s): Razavi, Michael
• Application Number:
  US07/378,626
• Time in Patent Office: 671 Days
• Field of Search: 382/58, 382/2, 382/1, 382/8
• US Class Current: 382/117
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/88 Image or video recognition using optical means, e.g. reference filters, holographic masks, frequency domain filters or spatial domain filters
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Characteristic features of images of an object eye are extracted to enable non-contact detection of eye movement. Two images of the eye are focused, and a differential image is generated to eliminate background noise and to permit feature extraction to be performed. In one feature of the invention, the illuminating light is polarized for use as a reference and the reflected light is separated to two light paths, each of which is focused to form an image of the object. In one path a polarizing plate blocks the regularly reflected light from the cornea so that only a diffused reflection component of the illuminating light from the other parts of the eye is passed, while in the other path both the regularly and diffusedly reflected light components are passed. A resulting differential image emphasizes the regular reflection component from the cornea relative to the background. In another aspect of the invention, two light sources are placed at different positions relative to the optical axis, to provide bright and dark images of the pupil without otherwise affecting the reflected image. A resulting differential image emphasizes the pupil relative to background noise.

Claims

1. An image pickup apparatus extracting features of an image of an object, comprising:

illuminating means (61,

     62) for illuminating said object with polarized light;

an optical system (65, 607, 608,

     609) for forming an image of reflected light out of the polarized light emitted by said illuminating means for illuminating said object;

a polarizing element (615,

     616) provided on an image forming optical axis of the optical system for blocking a regularly reflected component of said reflected light and for passing a diffused reflected component; and

image detecting means (612, 613,

     614) provided on said image forming point for detecting the diffused reflected component extracted by said polarizing element, wherein

said optical system comprises optical separating means (608,

     609) for separating the reflected light from said object into first and second optical paths to form images, respectively;

said polarizing element comprises an element (616) provided on an optical axis of the first optical path of said optical separating means;

said image detecting means comprises

first means (613) provided on the optical axis of the first optical path of said optical separating means for detecting the diffused reflected component extracted by said polarizing element, and

second means (64) provided on an optical axis of the second optical path of said optical separating means for detecting the regularly reflected component and the diffused reflected component out of said reflected light;

said image pickup apparatus further comprising;

image processing means (617) responsive to outputs from said first and second means for forming a differential image.

2. An image pickup apparatus according to claim 21, wherein

said illuminating means comprises first light sources (61) provided around an optical axis of said light separating means, each of said light sources emitting light of a first wavelength to said object;

a first polarizing element (63) provided on light emitting surfaces of said first light sources;

a second light source (62) provided apart from the optical axis of said light separating means for emitting light of a second wavelength to said object; and

a second polarizing element (67) provided on a light emitting surface of said second light source.

3. An image pickup apparatus according to claim 3, wherein

said light separating means comprises optical wavelength separating means (610) for separating reflected light of said first and second wavelengths from said object in accordance with respective wavelengths to form images;

said polarizing plate element located on an image forming optical axis of the reflected light having the first wavelength separated by said optical wavelength separating means;

said image generating means comprises

first means (613) provided on the image forming optical axis of said reflected light having the first wavelength separated by said optical wavelength separating means for detecting the diffused reflected image passed by said polarizing plate, and

second means (614) provided on an image forming optical axis of said reflected light having the second wavelength separated by said optical wavelength separating means for detecting the regularly reflected component and the diffused reflected component;

said image pickup apparatus further comprising

image processing means (617) responsive to detected outputs from said first and second means for forming said differential image.

4. An image pickup apparatus for extracting features of an image of an object, comprising:

illuminating means (61,

     62) for emitting polarized light to said object;

first image forming means (607) for separating a part of light reflected from said object to form an image;

second image forming means (609,

     610) separating a part of the light reflected from said object and blocking a polarized regularly reflected component for providing a non-polarized diffused reflected component to form an image;

image detecting means (612,

     613) provided at image forming points of said first and second image forming means, respectively; and

image processing means (618) for generating a differential image between an image formed by said first image forming means including the regularly reflected component detected by said image detecting means and an image formed by said second image forming means including only the diffused reflected component.

5. An image pickup apparatus for extracting features of an image of an object, comprising:

a taking lens (65) for picking up said object;

illuminating means (61) including a plurality of light sources (66) provided around an optical axis of said taking lens, each said light source arranged such that an angle formed by a segment coupling each said light source and the object and a segment coupling the center of said taking lens and the object is smaller than a prescribed value θ

th;

an optical system (607, 608,

     609) for forming an image from reflected light from said object through said taking lens; and

image detecting means (612,

     613) provided on an image forming point of said optical system for detecting a formed image.

6. An image pickup apparatus according to claim 6, wherein

said plurality of light sources comprise light emitting diodes.

7. An image pickup apparatus for extracting features of an image of an object, comprising:

illuminating means (113,

     115) having a light emitting diode;

driving means (110, 112,

     114) for turning on/off said light emitting diode;

an optical system (104) for forming an image from reflected light from said object; and

image detecting means (105,

     106) provided at an image forming point of said optical system for detecting a formed image;

wherein

said driving means includes means for intermittently driving said light emitting diode in synchronization with timing of detection of said image detecting means as needed.

8. An image pickup apparatus for extracting from an image of a face a characteristic portion necessary for eye tracking, comprising:

illuminating means for emitting polarized light to a face which is an object to be imaged;

light separating means for separating a reflected portion of the emitted polarized light reflected by said object to be imaged to first and second light paths and for respectively focusing each path to form respective images of the subject;

a polarizing plate provided on one of the light paths separated by said light separating means for blocking a regular reflection component of said reflected portion of the polarized light and for passing a diffused reflection component thereof; and

image generating means for generating a differential image of the respective images focused by said light separating means.

9. An image pickup apparatus for extracting from an image of a face a characteristic portion necessary for eye tracking, comprising:

a first light source provided near an optical axis of a photographing lens, a first angle defined between a line segment connecting the first light source and an eyeball which is an object to be photographed and a line segment connecting the center of said photographing lens and the object, said first angle being smaller than a prescribed value θ

th₁ ;

a second light source spaced apart from the optical axis of said lens by a prescribed distance, a second angle defined between a line segment connecting the second light source and the object and a line segment connecting the center of said photographing lens and said object, said second angle being larger than a prescribed value θ

th₂ ;

an optical system for separating light from said first light source reflected by said object and light from said second light source reflected by said object and for respectively focusing said light from said first and second light sources reflected by said object to form respective images thereof; and

image generating means for generating a differential image of said respective images focused by said optical system.

10. An image pickup apparatus as recited in claim 22, wherein said first light source includes means for illuminating a pupil in the eyeball to be photographed to provide a bright image of the pupil, and

said second light source includes means for illuminating a pupil in the eyeball to be photographed to provide a dark image of the pupil while illuminating background features at a level of illumination substantially similar to illumination thereof provided by said first light source means,

whereby said differential image generated by said image generating means substantially eliminates images of background features, thereby emphasizing an image of the pupil and distinguishing the pupil from images of background features.

11. An image pickup apparatus as recited in claim 23, wherein said first light source includes polarizing means thereby to emit polarized light to the eyeball to be photographed;

and further comprising

light separating means for separating a reflected portion of the polarized light reflected by said object to be imaged to first and second light paths and for respectively focusing each path to form respective images of the eyeball;

a polarizing plate provided on one of the light paths separated by said light separating means for blocking a regular reflection component of said reflected portion of the polarized light and for passing a diffused reflection component thereof; and

second image generating means for generating a second differential image of the respective images focused by said light separating means,

whereby said second differential image generated by said second image generating means substantially eliminates images of background features, thereby emphasizing an image of a cornea of the eyeball and distinguishing the cornea from images of background features.