GAZE DETECTION APPARATUS, GAZE DETECTION COMPUTER PROGRAM, AND DISPLAY APPARATUS

• US 20140085451A1
• Filed: 07/22/2013
• Published: 03/27/2014
• Est. Priority Date: 09/24/2012
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: KAMIMURA, Takuya, YOSHIKAWA, Hiroyasu
• Granted Patent: US 9,473,696 B2
• Time in Patent Office: Days
• US Class Current: 348/78
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

A gaze detection apparatus includes a light source which illuminates a user'"'"'s eye, an imaging unit which generates an image by capturing the image of a user'"'"'s face, and a processor adapted to: estimate a distance between the imaging unit and the user'"'"'s face; calculate the ratio of the distance between the imaging unit and the user'"'"'s face to a separation distance between the imaging unit and the light source and determine, based on the ratio, whether a bright pupil phenomenon occurs; detect from the captured image a corneal reflection image of the light source and a center of the user'"'"'s pupil when determining that the bright pupil phenomenon does not occur; and detect the user'"'"'s gaze direction or gaze position based on a positional relationship between the center of the user'"'"'s pupil and the corneal reflection image when determining that the bright pupil phenomenon does not occur.

Claims

1. A gaze detection apparatus comprising:

a light source which illuminates a user'"'"'s eye;

an imaging unit which generates an image by capturing the image of a user'"'"'s face; and

a processor adapted to;

estimate a distance between the imaging unit and the user'"'"'s face;

calculate the ratio of the distance between the imaging unit and the user'"'"'s face to a separation distance between the imaging unit and the light source and determine, based on the ratio, whether a bright pupil phenomenon in which an entire pupil of the user'"'"'s eye appears bright due to illumination from the light source occurs;

detect from the captured image a corneal reflection image of the light source and a center of the user'"'"'s pupil when it is determined that the bright pupil phenomenon does not occur; and

detect the user'"'"'s gaze direction or gaze position based on a positional relationship between the center of the user'"'"'s pupil and the corneal reflection image when it is determined that the bright pupil phenomenon does not occur.

2. The gaze detection apparatus according to claim 1, wherein determining whether or not the bright pupil phenomenon occurs includes comparing the ratio of the distance between the imaging unit and the user'"'"'s face to the separation distance between the imaging unit and the light source with a lower limit value thereof at and above which the bright pupil phenomenon occurs, and wherein when the ratio is not smaller than the lower limit value, it is determined that the bright pupil phenomenon occurs, on the other hand, when the ratio is smaller than the lower limit value, it is determined that the bright pupil phenomenon does not occur.

3. The gaze detection apparatus according to claim 2, wherein the separation distance between the imaging unit and the light source is set so that when the ratio of the distance between the imaging unit and the user'"'"'s face to the separation distance is equal to the lower limit value, the size of the user'"'"'s pupil and the size of the corneal reflection image are both at a lower limit of recognizable size on the captured image.

4. The gaze detection apparatus according to claim 1, further comprising a second light source which is mounted in a position spaced farther away from the imaging unit than the first light source is, wherein

determining whether a bright pupil phenomenon occurs includes, when it is determined that the bright pupil phenomenon occurs, turning off the first light source and turning on the second light source, and

the imaging unit generates a second image by capturing the image of the user'"'"'s face with the second light source turned on, and wherein

detecting the corneal reflection image of the light source and the center of the user'"'"'s pupil includes detecting the corneal reflection image of the second light source and the center of the user'"'"'s pupil from the second image, and

detecting the user'"'"'s gaze direction or gaze position includes detecting the user'"'"'s gaze direction or gaze position based on the positional relationship between the corneal reflection image of the second light source and the center of the user'"'"'s pupil.

5. The gaze detection apparatus according to claim 1, wherein the processor is further adapted to detect the center of the user'"'"'s cornea from the captured image and detect the user'"'"'s gaze direction or gaze position based on the center of the user'"'"'s cornea when it is determined that the bright pupil phenomenon occurs.

6. The gaze detection apparatus according to claim 1, further comprising a second imaging unit which is mounted in a different position than the first imaging unit and which generates a third image by capturing the image of the user'"'"'s face, and wherein

estimating the distance includes estimating the distance between the imaging unit and the user'"'"'s face based on the positional relationship between the position of the user'"'"'s eye on the image and the position of the user'"'"'s eye on the third image.

7. The gaze detection apparatus according to claim 1, wherein estimating the distance includes detecting the pupils of the user'"'"'s left and right eyes, respectively, on the image and estimating the distance between the imaging unit and the user'"'"'s face based on the separation distance between the pupils of the left and right eyes on the image.

8. A gaze detection method comprising:

estimating a distance between a user'"'"'s face and an imaging unit which generates an image by capturing the image of the user'"'"'s face by a processor;

calculating the ratio of the distance between the imaging unit and the user'"'"'s face to a separation distance between the imaging unit and a first light source which illuminates the user'"'"'s face, and determining, based on the ratio, whether a bright pupil phenomenon in which an entire pupil of the user'"'"'s eye appears bright due to illumination from the first light source occurs by the processor; and

when it is determined by the processor that the bright pupil phenomenon does not occur, detecting from the captured image a corneal reflection image of the first light source and a center of the user'"'"'s pupil and detecting the user'"'"'s gaze direction or gaze position based on a positional relationship between the center of the user'"'"'s pupil and the corneal reflection image by the processor.

9. The gaze detection method according to claim 8, wherein when it is determined that the bright pupil phenomenon occurs, turning off the first light source and turning on a second light source, which is mounted in a position spaced farther away from the imaging unit than the first light source is and wherein

detecting the corneal reflection image of the light source and the center of the user'"'"'s pupil includes detecting the corneal reflection image of the second light source and the center of the user'"'"'s pupil from a second image generated by the imaging unit capturing the image of the user'"'"'s face with the second light source turned on, and

detecting the user'"'"'s gaze direction or gaze position includes detecting the user'"'"'s gaze direction or gaze position based on the positional relationship between the corneal reflection image of the second light source and the center of the user'"'"'s pupil.

10. The gaze detection method according to claim 8, wherein estimating the distance includes estimating the distance between the imaging unit and the user'"'"'s face based on the positional relationship between the position of the user'"'"'s eye on the image and the position of the user'"'"'s eye on a third image generated by a second imaging unit capturing the image of the user'"'"'s face, the second imaging unit being mounted in a different position than the first imaging unit.

11. A non-transitory computer-readable recording medium having recorded thereon a gaze detection computer program for causing a computer to implement:

estimating a distance between a user'"'"'s face and an imaging unit which generates an image by capturing the image of the user'"'"'s face;

calculating the ratio of the distance between the imaging unit and the user'"'"'s face to a separation distance between the imaging unit and a light source which illuminates the user'"'"'s face, and determining, based on the ratio, whether a bright pupil phenomenon in which an entire pupil of the user'"'"'s eye appears bright due to illumination from the light source occurs; and

when it is determined that the bright pupil phenomenon does not occur, detecting from the captured image a corneal reflection image of the light source and a center of the user'"'"'s pupil and detecting the user'"'"'s gaze direction or gaze position based on a positional relationship between the center of the user'"'"'s pupil and the corneal reflection image.

12. A display apparatus comprising an illuminating light source and an infrared imaging unit, wherein the illuminating light source and the infrared imaging unit are separated by a minimum distance determined based on a maximum expected distance between the display apparatus and a user thereof and on a prescribed coefficient.

13. The display apparatus according to claim 12, wherein the prescribed coefficient represents a lower limit value of the ratio of the distance between the infrared imaging unit and the user'"'"'s face to the separation distance between the illuminating light source and the infrared imaging unit, at and above which bright pupil phenomenon can occur, and wherein the separation distance is not smaller than the maximum expected distance divided by the lower limit value of the ratio.

14. The display apparatus according to claim 12, wherein the maximum expected distance is a distance at which a size of the user'"'"'s pupil and a size of the corneal reflection image are both at a lower limit of recognizable size on an image generated by the infrared imaging unit.