GAZE DETECTION APPARATUS AND GAZE DETECTION METHOD

• US 20140055342A1
• Filed: 06/04/2013
• Published: 02/27/2014
• Est. Priority Date: 08/21/2012
• Status: Abandoned Application

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: KAMIMURA, Takuya, YOSHIKAWA, Hiroyasu
• Application Number:
  US13/909,452
• Publication Number:
  US 20140055342A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A gaze detection apparatus includes: a first imaging unit which has a first angle of view and generates a first image; a second imaging unit which has a second angle of view narrower than the first angle of view, and generates a second image; a face detection unit which detects from the first image a face region; a coordinate conversion unit which identifies on the second image a first region corresponding to the face region or to an eye peripheral region containing the user'"'"'s eye; a Purkinje image detection unit which detects a corneal reflection image of a light source and the center of the user'"'"'s pupil from within an eye region, identified based on the first region; and a gaze detection unit which detects the user'"'"'s gaze direction or gaze position based on a positional relationship between the center of the pupil and the corneal reflection image.

Claims

1. A gaze detection apparatus comprising:

a light source which illuminates a user'"'"'s eye;

a first imaging unit which has a first angle of view, and generates a first image by capturing an image of the user'"'"'s face;

a second imaging unit which has a second angle of view narrower than the first angle of view, and generates a second image by capturing an image of at least a portion of the user'"'"'s face;

a face detection unit which detects from the first image a face region containing the user'"'"'s face;

a coordinate conversion unit which identifies on the second image a first region that corresponds to the face region or to an eye peripheral region detected from within the face region as containing the user'"'"'s eye;

a Purkinje image detection unit which detects a corneal reflection image of the light source and the center of the user'"'"'s pupil from within an eye region, identified based on the first region, that contains the user'"'"'s eye on the second image; and

a gaze detection unit which detects the user'"'"'s gaze direction or gaze position based on a positional relationship between the center of the pupil and the corneal reflection image.

2. The gaze detection apparatus according to claim 1, further comprising an eye peripheral region detection unit which detects the eye peripheral region from within the face region detected on the first image.

3. The gaze detection apparatus according to claim 2, wherein the first region is a region on the second image that corresponds to the eye peripheral region, and further comprising an eye precision detection unit which detects the eye region within a search range that is set on the second image in accordance with the first region.

4. The gaze detection apparatus according to claim 3, wherein when the whole of the first region is contained in the second image, the eye precision detection unit detects the eye region by using information corresponding to both eyes of the user, while, when a portion of the first region is not contained in the second image, the eye precision detection unit detects the eye region by using information corresponding to one or the other eye of the user and the user'"'"'s face parts other than the eye.

5. The gaze detection apparatus according to claim 4, wherein when the whole of the first region is contained in the second image, the eye precision detection unit sets the first region as the search range, while, when a portion of the first region is not contained in the second image, the eye precision detection unit sets the search range by taking the first region and a region that is located around the first region and that potentially contains the user'"'"'s face parts other than the eye.

6. The gaze detection apparatus according to claim 3, wherein the first imaging unit and the second imaging unit are arranged vertically spaced apart from each other, and the eye precision detection unit sets the search range in such a manner as to be bounded by left and right edges of the first region.

7. The gaze detection apparatus according to claim 1, wherein the first region is a region on the second image that corresponds to the face region, and further comprising an eye precision detection unit which detects the eye region within a second search range that is set on the second image in accordance with the first region.

8. The gaze detection apparatus according to claim 7, further comprising a face precision detection unit which detects a second face region containing the user'"'"'s face, within the second search range that is set on the second image in accordance with the first region, and wherein the eye precision detection unit detects the eye region from within the second face region.

9. The gaze detection apparatus according to claim 8, wherein the first imaging unit and the second imaging unit are arranged vertically spaced apart from each other, and the face precision detection unit sets the second search range in such a manner as to be bounded by left and right edges of the first region.

10. The gaze detection apparatus according to claim 1, wherein the first imaging unit and the second imaging unit are disposed around a display screen of a display device, and wherein

the gaze detection unit detects the user'"'"'s gaze position on the display screen, based on the positional relationship between the center of the pupil and the corneal reflection image.

11. The gaze detection apparatus according to claim 10, wherein the first imaging unit is disposed above the display screen, and the second imaging unit is disposed below the display screen.

12. The gaze detection apparatus according to claim 1, wherein the first imaging unit and the second imaging unit are disposed around a display screen of a display device, and wherein

the gaze detection unit estimates a distance from the display device to the user'"'"'s face, based on the position of the user'"'"'s eye in the first region on the first image and the position of the user'"'"'s eye in the eye region on the second image and on the distance between the first imaging unit and the second imaging unit, and detects the user'"'"'s gaze position on the display screen, based on the estimated distance and on the positional relationship between the center of the pupil and the corneal reflection image.

13. A gaze detection method comprising:

detecting a face region containing a user'"'"'s face from a first image generated by a first imaging unit having a first angle of view by a processor;

identifying a first region on a second image generated by a second imaging unit having a second angle of view narrower than the first angle of view by the processor, wherein the first region corresponds to the face region or to an eye peripheral region detected from within the face region as containing the user'"'"'s eye;

detecting a corneal reflection image of a light source illuminating the user'"'"'s eye and the center of the user'"'"'s pupil from within an eye region, identified based on the first region, that contains the user'"'"'s eye on the second image by the processor; and

detecting the user'"'"'s gaze direction or gaze position based on a positional relationship between the center of the pupil and the corneal reflection image by the processor.

14. A non-transitory computer-readable recording medium having recorded thereon a gaze detection computer program for causing a computer to execute:

detecting a face region containing a user'"'"'s face from a first image generated by a first imaging unit having a first angle of view;

identifying a first region on a second image generated by a second imaging unit having a second angle of view narrower than the first angle of view, wherein the first region corresponds to the face region or to an eye peripheral region detected from within the face region as containing the user'"'"'s eye;

detecting a corneal reflection image of a light source illuminating the user'"'"'s eye and the center of the user'"'"'s pupil from within an eye region, identified based on the first region, that contains the user'"'"'s eye on the second image; and

detecting the user'"'"'s gaze direction or gaze position based on a positional relationship between the center of the pupil and the corneal reflection image.