Eye-gaze detecting device and eye-gaze detecting method

• US 8,259,169 B2
• Filed: 02/25/2009
• Issued: 09/04/2012
• Est. Priority Date: 02/28/2008
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Sugio, Toshiyasu, Sato, Daisuke
• Primary Examiner(s): Maung, Zarni
• Application Number:
  US12/597,264
• Publication Number:
  US 20100165093A1
• Time in Patent Office: 1,287 Days
• Field of Search: 348/78, 348/135
• US Class Current: 348/78
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G03B 15/00 Special procedures for taking photographs; Apparatus therefor
• CPC Class Codes: G03B 2213/025 Sightline detection
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Accurately and quickly calculated is a calibrating parameter for detecting an eye-gaze coordinate, of an operator, in an input image from a capturing unit worn on a user.

An eye-gaze detecting device (100) detects an eye-gaze coordinate of a user in an input image from a capturing unit (104) wearable on the user, and includes: an eye movement detecting unit (101) detecting an eye movement of the user; a calibration object detecting unit (105) (i) receiving the input image, (ii) determining whether or not there is an object available as a calibration object in the input image, and (iii) providing a position coordinate indicating a position of the object in the case where the available object is determined to be included in the input image; a voice instructing unit (108) providing an instruction to the user to look at the calibration object indicated in the position coordinate; a calibration parameter calculating unit (106) calculating a calibration parameter with a use of the eye movement information of the user receiving the instruction and the position coordinate of the calibration object; and a calibrating unit (102) calibrating the eye movement information to the eye-gaze coordinate with a use of the calculated calibration parameter.

Claims

1. An eye-gaze detecting device which detects an eye-gaze coordinate of a user in an input image from a capturing unit wearable on the user, said eye-gaze detecting device comprising:

an eye movement detecting unit configured to detect an eye movement of the user to obtain eye movement information indicating the eye movement;

a calibration object detecting unit configured to (i) receive the input image, (ii) determine whether or not there is an object available as a calibration object in the input image, and (iii) provide a position coordinate indicating a position of the object in the case where the available object is determined to be included in the input image;

an instructing unit configured to provide an instruction to the user to look at the calibration object indicated in the position coordinate;

a calibration parameter calculating unit configured to calculate a calibration parameter with a use of the eye movement information of the user receiving the instruction from said instructing unit, and the position coordinate of the calibration object, the calibration parameter being used for calibrating the eye movement information to the eye-gaze coordinate; and

a calibrating unit configured to calibrate the eye movement information to the eye-gaze coordinate with a use of the calibration parameter calculated by said calibration parameter calculating unit.

2. The eye-gaze detecting device according to claim 1,

wherein said calibration object detecting unit is further configured to provide a completion signal when determining that the available object is included in the input image, the completion signal indicating that the detection of the calibration object has been completed, and

said calibration parameter calculating unit is configured to calculate the calibration parameter when receiving the completion signal, the calibration parameter being calculated with a use of the eye movement information and the position coordinate of the calibration object.

3. The eye-gaze detecting device according to claim 2,

wherein said calibration object detecting unit is configured to (i) extract an amount of image characteristics of the input image, (ii) determine whether or not there is an area having the extracted amount of image characteristics reaching a predetermined threshold, and (iii), when determining that there is the area, provide a position coordinate of the area as the position coordinate of the object available as the calibration object.

4. The eye-gaze detecting device according to claim 3,

wherein said calibration object detecting unit is further configured to provide the extracted amount of image characteristics to said instructing unit when determining that the area has been found, and

said instructing unit is configured to provide the instruction to the user to look at the calibration object indicated in the position coordinate and the amount of image characteristics.

5. The eye-gaze detecting device according to claim 4,

wherein said instructing unit is configured to produce navigation information on the calibration object, and provide the produced navigation information to the user as the instruction via voice or bone conduction, the calibration object being indicated in the position coordinate and the amount of image characteristics.

6. The eye-gaze detecting device according to claim 3,

wherein said calibration object detecting unit is configured to extract at least one of edge information, color information, movement information, and luminance level information as the amount of image characteristics of the input image.

7. The eye-gaze detecting device according to claim 2,

wherein said calibration object detecting unit is configured to determine whether or not a human face, part of a body of the user, or a belonging which the user is wearing or holding is included in the input image as the object available as the calibration object.

8. The eye-gaze detecting device according to claim 2, further comprising

a calibration object outputting unit configured to output the calibration object,

wherein said instructing unit is configured to provide the instruction to the user in order for the user to look at the calibration object outputted from said calibration object outputting unit.

9. The eye-gaze detecting device according to claim 2, further comprising

a distance-to-calibration-object measuring unit configured to measure distance to said calibration object,

wherein said calibration parameter calculating unit is configured to calculate the calibration parameter with a further use of the distance measured by said distance-to-calibration-object measuring unit.

10. The eye-gaze detecting device according to claim 2,

wherein said eye movement detecting unit is configured to obtain information indicating an eye potential or a movement amount of an eye of the user as the eye movement information.

11. The eye-gaze detecting device according to claim 2,

wherein the calibration parameter is a table associating the eye movement information of the user with the eye-gaze coordinate,

said calibration parameter calculating unit is configured to renew the table with a use of the eye movement information of the user and the position coordinate of the calibration object, the user receiving the instruction from said instructing unit, and

said calibrating unit is configured to calibrate the eye movement information to the eye-gaze coordinate by referring to the table renewed by said calibration parameter calculating unit.

12. An eye-gaze detecting method for detecting an eye-gaze coordinate of a user in an input image from a capturing unit wearable on the user, said eye-gaze detecting method comprising:

detecting an eye movement of the user to obtain eye movement information indicating the eye movement;

receiving the input image, determining whether or not there is an object available as a calibration object in the input image, and providing a position coordinate indicating a position of the object;

providing an instruction to the user to look at the calibration object indicated in the position coordinate in the case where the available object is determined to be included in the input image;

calculating a calibration parameter with a use of the eye movement information of the user receiving the instruction, and the position coordinate of the calibration object, the calibration parameter being used for calibrating the eye movement information to the eye-gaze coordinate; and

calibrating the eye movement information to the eye-gaze coordinate with a use of the calibration parameter calculated in said calculating.

13. A non-transitory computer readable medium storing a program causing a computer to execute the eye-gaze detecting method according to claim 12.

14. An integrated circuit which detects an eye-gaze coordinate of a user in an input image from a capturing unit wearable on the user, said integrated circuit comprising:

an eye movement detecting unit configured to detect an eye movement of the user to obtain eye movement information indicating the eye movement;

a calibration object detecting unit configured to (i) receive the input image, (ii) determine whether or not there is an object available as a calibration object in the obtained input image, and (iii) provide a position coordinate indicating a position of the object in the case where the available object is determined to be included in the input image;

an instructing unit configured to provide an instruction to the user to look at the calibration object indicated in the position coordinate;

a calibration parameter calculating unit configured to calculate a calibration parameter with a use of the eye movement information of the user receiving the instruction from said instructing unit, and the position coordinate of the calibration object, the calibration parameter being used for calibrating the eye movement information to the eye-gaze coordinate; and

a calibrating unit configured to calibrate the eye movement information to the eye-gaze coordinate with a use of the calibration parameter calculated by said calibration parameter calculating unit.