Gaze position detection apparatus and gaze position detection method

• US 10,146,306 B2
• Filed: 05/10/2017
• Issued: 12/04/2018
• Est. Priority Date: 06/06/2016
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Picard, Sebastien
• Primary Examiner(s): Lin, Chun-Nan
• Application Number:
  US15/591,756
• Publication Number:
  US 20170351329A1
• Time in Patent Office: 573 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A gaze position detection apparatus includes: a memory which stores a plurality of registered move patterns of a gaze position and correction information of the gaze position included in the registered move pattern for each of the registered move patterns; a move pattern detection unit which detects a move pattern of a user'"'"'s gaze position, from a plurality of measurement points of the user'"'"'s gaze position which are obtained from data items representing a user'"'"'s gaze direction generated by a gaze detection device in different timing; a similar-pattern extraction unit which extracts the registered move pattern similar to the move pattern of the user'"'"'s gaze position from among the plurality of registered move patterns; and a correction unit which corrects the gaze position included in the move pattern of the user'"'"'s gaze position using the correction information of the gaze position for the similar registered move pattern.

Claims

1. A gaze position detection apparatus comprising:

a memory configured to store a plurality of registered move patterns of a gaze position and correction information of the gaze position included in the registered move pattern for each of the plurality of registered move patterns; and

a processor configured to;

detect a move pattern of a user'"'"'s gaze position, from a plurality of measurement points of the user'"'"'s gaze position which are obtained from data items representing a user'"'"'s gaze direction generated by a gaze detection device in different timing;

extract at least one feature among a first feature related to the user, a second feature related to an arrangement of the gaze detection device, and a third feature related to the environment at the time when the move pattern is detected;

extract the registered move pattern similar to the move pattern from among the plurality of registered move patterns; and

correct the gaze position included in the move pattern using the correction information of the gaze position for the similar registered move pattern, wherein the memory stores the at least one feature among the first feature, the second feature, and the third feature, for each of the plurality of registered move patterns, and

the processor selects, among the plurality of registered move patterns, one or more registered move patterns, a difference between the at least one feature for the move pattern and the at least one feature for the registered move pattern being within a predetermined allowable range, and extracts the similar registered move pattern among from the selected registered move patterns.

2. The gaze position detection apparatus according to claim 1, wherein the first feature includes at least one of pupil size, head posture, or moving speed of the gaze.

3. The gaze position detection apparatus according to claim 1, wherein the gaze detection device includes a light source and a camera, and the second feature includes at least one of positional relationship between the gaze detection device and a target surface at which the user is to gaze, positional relationship between the light source and the camera, an angle of view of the camera, information representing contents displayed on the surface, and identification information of an apparatus including the surface.

4. The gaze position detection apparatus according to claim 1, wherein the third feature includes at least one of illumination intensity at the time when the move pattern is detected, time and date of the detection of the move pattern, and information representing an installation position at which the gaze detection device is installed.

5. The gaze position detection apparatus according to claim 1, wherein every time a predetermined time period elapsed after one or more fixation points are detected from a plurality of measurement points of user'"'"'s gaze positions, the processor sets a series of fixation points obtained in the predetermined time period as the move pattern.

6. The gaze position detection apparatus according to claim 5, wherein the processor determines that the predetermined time period elapsed every time a predetermined number of fixation points are detected.

7. The gaze position detection apparatus according to claim 5, wherein a plurality of partial areas are displayed on the target surface at which the user is to gaze, and the processor determines that the predetermined time period elapsed when a total of move distances between the one or more fixation points reaches or exceeds a value obtained by multiplying an average of distances between adjacent partial areas among the plurality of partial areas by a certain number.

8. A gaze position detection method including:

detecting a move pattern of a user'"'"'s gaze position, from a plurality of measurement points of the user'"'"'s gaze position which are obtained from data items representing a user'"'"'s gaze direction generated by a gaze detection device in different timing;

extracting at least one feature among a first feature related to the user, a second feature related to an arrangement of the gaze detection device, and a third feature related to the environment at the time when the move pattern is detected;

extracting the registered move pattern similar to the move pattern from among the plurality of registered move patterns of a gaze position stored in a storage unit; and

correcting the gaze position included in the move pattern using correction information of the gaze position corresponding to the similar registered move pattern;

wherein

extraction of the registered move pattern includes selecting, among the plurality of registered move patterns, one or more registered move patterns, a difference between the at least one feature for the move pattern and the at least one feature for the registered move pattern being within a predetermined allowable range, and extracting the similar registered move pattern among from the selected registered move patterns.

9. The gaze position detection method according to claim 8, wherein the first feature includes at least one of pupil size, head posture, or moving speed of the gaze.

10. The gaze position detection method according to claim 8, wherein the gaze detection device includes a light source and a camera, and the second feature includes at least one of positional relationship between the gaze detection device and a target surface at which the user is to gaze, positional relationship between the light source and the camera, an angle of view of the camera, information representing contents displayed on the surface, and identification information of an apparatus including the surface.

11. The gaze position detection method according to claim 8, wherein the third feature includes at least one of illumination intensity at the time when the move pattern is detected, time and date of the detection of the move pattern, and information representing an installation position at which the gaze detection device is installed.

12. The gaze position detection method according to claim 8, wherein detection of the move pattern of the user'"'"'s gaze position includes, every time a predetermined time period elapsed after one or more fixation points are detected from a plurality of measurement points of user'"'"'s gaze positions, setting a series of fixation points obtained in the predetermined time period as the move pattern.

13. The gaze position detection method according to claim 12, further comprising:

determining that the predetermined time period elapsed every time a predetermined number of fixation points are detected.

14. The gaze position detection method according to claim 12, wherein a plurality of partial areas are displayed on the target surface at which the user is to gaze, and further comprising:

determining that the predetermined time period elapsed when a total of move distances between the one or more fixation points reaches or exceeds a value obtained by multiplying an average of distances between adjacent partial areas among the plurality of partial areas by a certain number.

15. A non-transitory computer-readable recording medium having recorded thereon a computer program for detecting a gaze position that causes a computer to execute a process comprising:

detecting a move pattern of a user'"'"'s gaze position, from a plurality of measurement points of the user'"'"'s gaze position which are obtained from data items representing a user'"'"'s gaze direction generated by a gaze detection device in different timing;

extracting at least one feature among a first feature related to the user, a second feature related to an arrangement of the gaze detection device, and a third feature related to the environment at the time when the move pattern is detected;

extracting the registered move pattern similar to the move pattern from among the plurality of registered move patterns of a gaze position stored in a storage unit; and

correcting the gaze position included in the move pattern using correction information of the gaze position corresponding to the similar registered move pattern;

wherein

extraction of the registered move pattern includes selecting, among the plurality of registered move patterns, one or more registered move patterns, a difference between the at least one feature for the move pattern and the at least one feature for the registered move pattern being within a predetermined allowable range, and extracting the similar registered move pattern among from the selected registered move patterns.