Line-of-sight detection apparatus

• US 9,760,774 B2
• Filed: 07/14/2015
• Issued: 09/12/2017
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: Alps Alpine Co., Ltd.
• Original Assignee: ALPS Electric Company Limited (Alps Alpine Co., Ltd.)
• Inventors: Yamashita, Tatsumaro
• Primary Examiner(s): Adams, Eileen
• Application Number:
  US14/799,193
• Publication Number:
  US 20160063304A1
• Time in Patent Office: 791 Days
• Field of Search: None
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

In computation of a line-of-sight direction in a control unit, a first computed line-of-sight direction is obtained from the center of a pupil and a reflection point of cornea reflection light, a second computed line-of-sight direction is obtained by computing the ellipse shape of an iris or the pupil, and a third computed line-of-sight direction is obtained on the basis of a face orientation and the relative position of the iris or pupil in an eye. The line-of-sight direction can be computed over a wide range with high accuracy by selecting one of the computed line-of-sight directions.

Claims

1. A line-of-sight detection apparatus comprising:

a plurality of light sources that radiate detection light to a face of a target person;

a camera that captures an image of the face of the target person; and

a control unit that computes a line-of-sight direction on a basis of the image captured by the camera,

wherein the control unit performs;

an extraction step of extracting partial images of respective different portions of the face on the basis of the image captured by the camera;

a computation step of computing line-of-sight directions on a basis of the respective partial images; and

a selection step of selecting one of a plurality of computed line-of-sight directions that are obtained in different ways and from different partial images, in accordance with angle ranges of the line-of-sight directions and

wherein;

the partial images include a pupil image, a cornea reflection light image, and an iris image,

a first computed line-of-sight direction is obtained from the pupil image and the cornea reflection light image,

a second computed line-of-sight direction is obtained from a shape of the pupil image or the iris image,

the first computed line-of-sight direction is selected as a correct computed line-of-sight direction while an angle of the line-of-sight direction with respect to a reference value is within a first angle range, and the second computed line-of-sight direction is selected as the correct computed line-of-sight direction when the angle of the line-of-sight direction has exceeded the first angle range,

the partial images are the face image and the iris image or are the face image and the pupil image, the face orientation is computed from the face image, a position of the iris in an eye or a position of the pupil in the eye is computed from the iris image or the pupil image, and a third computed line-of-sight direction is computed from the face orientation and the position of the iris or from the face orientation and the position of the pupil, and

the third computed line-of-sight direction is selected as the correct computed line-of-sight direction when an angle of the line-of-sight direction has exceeded a second angle range which includes larger angles than the first angle range.

2. A line-of-sight detection apparatus comprising:

a plurality of light sources that radiate detection light to a face of a target person;

a camera that captures an image of the face of the target person; and

a control unit that computes a line-of-sight direction on a basis of the image captured by the camera,

wherein the control unit performs;

an extraction step of extracting partial images of respective different portions of the face on the basis of the image captured by the camera;

a computation step of computing line-of-sight directions on a basis of the respective partial images; and

a selection step of selecting, from among a plurality of computed line-of-sight directions that are obtained in different ways and from different partial images, one having a small variation as a correct computed line-of-sight direction; and

wherein;

an angle of the line-of-sight direction is classified into a plurality of angle ranges, and a partial image to be selected to obtain a computed line-of-sight direction is assigned in advance to each angle range, and

in a border area of an angle range, from among different computed line-of-sight directions used in angle ranges on two sides of the border area, a computed line-of-sight direction having a smaller variation is selected as the correct computed line-of-sight direction.

3. The line-of-sight detection apparatus according to claim 2,

wherein the partial images include a pupil image, a cornea reflection light image, and an iris image,

wherein a first computed line-of-sight direction is obtained from the pupil image and the cornea reflection light image,

wherein a second computed line-of-sight direction is obtained from a shape of the pupil image or the iris image, and

wherein, one of the first and second computed line-of-sight directions that has a smaller variation is selected as the correct computed line-of-sight direction.

4. The line-of-sight detection apparatus according to claim 3,

wherein the partial images are the face image and the iris image or are the face image and the pupil image, the face orientation is computed from the face image, a position of the iris in an eye or a position of the pupil in the eye is computed from the iris image or the pupil image, and a third computed line-of-sight direction is computed from the face orientation and the position of the iris or from the face orientation and the position of the pupil, and

wherein, one of the second and third computed line-of-sight directions that has a smaller variation is selected as the correct computed line-of-sight direction when the first computed line-of-sight direction becomes unavailable.

5. A line-of-sight detection apparatus comprising:

a plurality of light sources that radiate detection light to a face of a target person;

a camera that captures an image of the face of the target person; and

a control unit that computes a line-of-sight direction on a basis of the image captured by the camera,

wherein the control unit performs;

an extraction step of extracting partial images of respective different portions of the face on the basis of the image captured by the camera;

a computation step of computing line-of-sight directions on a basis of the respective partial images; and

a selection step of selecting, from among a plurality of computed line-of-sight directions that are obtained in different ways and from different partial images, one having a small variation as a correct computed line-of-sight direction; and

wherein;

the partial images include a pupil image, a cornea reflection light image, and an iris image,

a first computed line-of-sight direction is obtained from the pupil image and the cornea reflection light image,

a second computed line-of-sight direction is obtained from a shape of the pupil image or the iris image,

one of the first and second computed line-of-sight directions that has a smaller variation is selected as the correct computed line-of-sight direction,

the partial images are the face image and the iris image or are the face image and the pupil image, the face orientation is computed from the face image, a position of the iris in an eye or a position of the pupil in the eye is computed from the iris image or the pupil image, and a third computed line-of-sight direction is computed from the face orientation and the position of the iris or from the face orientation and the position of the pupil, and

one of the second and third computed line-of-sight directions that has a smaller variation is selected as the correct computed line-of-sight direction when the first computed line-of-sight direction becomes unavailable.