DETECTION OF GAZE POINT ASSISTED BY OPTICAL REFERENCE SIGNAL

• US 20110279666A1
• Filed: 01/26/2009
• Published: 11/17/2011
• Est. Priority Date: 01/26/2009
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Gustafsson, Simon, Strömbom, Johan, Skogo, Marten, Holtz Elvesjö, John Mikael, Nystedt, Per, Blixt, Peter
• Granted Patent: US 9,495,589 B2
• Time in Patent Office: Days
• US Class Current: 348/78
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A gaze-point detection system includes at least one infrared (IR) signal source to be placed in a test scene as a reference point, a pair of eye glasses to be worn by a person, and a data processing and storage unit for calculating a gaze point of the person wearing the pair of eye glasses. The pair of eye glasses includes an image sensor, an eye-tracking unit and a camera. The image sensor detects IR signals from the at least one IR signal source and generates an IR signal source tracking signal. The eye-tracking unit determines adapted to determine the gaze direction of the person and generates an eye-tracking signal, and the camera acquires a test scene picture. The data processing and storage unit communicates with the pair of eye glasses and calculates the gaze point relative to the test scene picture.

Claims

1-35. -35. (canceled)

36. A pair of eye glasses comprising:

a frame;

an image sensor carried by said frame and configured to receive incident infrared (IR) signals from at least one IR signal source in a region of space, and to generate at least one IR signal source tracking signal; and

an eye-tracking unit carried by said frame and comprising

at least one IR-light-emitting device configured to emit a light pulse toward an eye of a person wearing the pair of eye glasses,

a detector configured to receive a light pulse reflected from the eye and to repeatedly acquire an image of the eye, and

an evaluation unit connected to said detector and configured to determine a gaze direction of the eye based on information extracted from the acquired image of the eye including location of a reflection on a cornea of the eye from the emitted light pulse and position of a pupil of the eye.

37. The pair of eye glasses according to claim 36, further comprising a camera carried by said frame and configured to acquire at least one picture of the region of space.

38. The pair of eye glasses according to claim 37, for use in a system comprising a data processing and storage unit configured to receive the at least one IR signal source tracking signal, the eye-tracking signal and the at least one picture, said data processing and storage unit configured to calculate a gaze point of the person with respect to the at least one picture based on the at least one IR signal source tracking signal, the eye-tracking signal and the at least one picture.

39. A pair of eye glasses according to claim 36, wherein said image sensor comprises a sensor surface and an optical aperture configured to receive incident IR signals, with said image sensor being configured to calculate at least one angle between an incident IR signal and an axis of the optical aperture by determining a position of the incident IR signal on said sensor surface, with the calculated angle being included in the IR signal source tracking signal.

40. The pair of eye glasses according to claim 36, wherein said at least one IR-light-emitting device and said detector are arranged non-coaxially.

41. The pair of eye glasses according to claim 36, further comprising a first polarization filter to be positioned in front of the eye and configured to pass through polarized light in a first direction, and a second polarization filter positioned in front of said detector and configured to pass through polarized light in a second direction that is essentially orthogonal to the first direction.

42. The pair of eye glasses according to claim 38, wherein said image sensor and said eye-tracking unit are communicatively coupled to a support unit comprising at least one of the following:

a voltage source, a data buffer, a data pre-processor, and a transceiver.

43. The pair of eye glasses according to claim 36, wherein said image sensor comprises:

a first optical line sensor having a scanning direction that deviates slightly from a vertical direction when the pair of eye glasses are worn; and

a second optical line sensor having a scanning direction that is substantially perpendicular to that of said first optical line sensor;

with scanning directions of said first and second optical line sensors both being substantially parallel to a frontal plane of the person wearing the pair of eye glasses.

44. A system for determining a gaze point of a person relative to a picture of a region of space, the system comprising:

at least one infrared (IR) signal source located in the region of space and configured to emit an IR signal;

a pair of eye glasses to be worn by a person, and comprising

a frame;

an image sensor carried by said frame and configured to receive incident IR signals from said at least one IR signal source in the region of space, and to generate at least one IR signal source tracking signal, and

an eye-tracking unit carried by said frame and comprising

at least one IR-light-emitting device configured to emit a light pulse directed toward an eye of the person wearing said pair of glasses,

a detector configured to receive a light pulse reflected from the eye and to repeatedly acquire an image of the eye, and

an evaluation unit connected to said detector and configured to determine a gaze direction of the eye based on information extracted from the acquired image including location of a reflection on a cornea of the eye from the emitted light pulses and position of a pupil of the eye; and

a data processing and storage unit configured to communicate with said pair of eye glasses to obtain the at least one IR signal source tracking signal and the eye-tracking signal, and to calculate the gaze point of the person with respect to the picture of the region of space based on the at least one IR signal source tracking signal, the eye-tracking signal and the picture.

45. The system according to claim 44, further comprising a scene camera configured to acquire at least one picture of the region of space.

46. The system according to claim 44, wherein said at least one pair of eye glasses further comprises a camera carried by said frame and configured to acquire at least one picture of the region of space.

47. The system according to claim 46, wherein said data processing and storage unit is configured to match a picture acquired by said camera to information from the at least one IR signal source tracking signal to create a combined image of the acquired picture and a position relative thereto of said at least one IR signal source.

48. The system according to claim 44, wherein said at least one IR signal source comprises at least four IR signal sources;

and wherein said data processing and storage unit is configured to further match the picture to information on positions of said at least four IR signal sources, and to store information of the positions of said at least four IR signal sources.

49. The system according to claim 44, wherein said data processing and storage unit is further configured to extract and continuously track image features in the acquired picture to detect a change of position of said pair of eye glasses.

50. The system according to claim 44, wherein the IR signal comprises a modulated IR light;

and wherein said data processing and storage unit is configured to determine an identity of a specific IR signal source based on the at least one IR signal source tracking signal.

51. The system according to claim 50, wherein the modulated IR light is modulated based on at least one of time, wavelength and polarization.

52. The system according to claim 44, wherein said image sensor comprises a sensor surface configured to receive the incident IR signals, with said image sensor being configured to calculate at least one angle between an incident IR signal and an axis orthogonal to the sensor surface and passing through a center of an optical aperture of said image sensor by determining a position of the incident IR signal on the sensor surface, with the at least one calculated angle being included in the IR signal source tracking signal.

53. The system according to claim 44, wherein said at least one IR-light-emitting device and said detector are arranged non-coaxially.

54. The system according to claim 53, wherein said pair of eye glasses further comprises a first polarization filter to be positioned in front of the eye and configured to pass through polarized light in a first direction, and a second polarization filter positioned in front of said detector and configured to pass through polarized light in a second direction that is essentially orthogonal to the first direction.

55. The system according to claim 44, further comprising a support unit communicatively coupled to said pair of eye glasses and comprising at least one of the following:

a voltage source, a data buffer, a data pre-processor, and a transceiver configured to communicate with said data processing and storage unit and to communicate with said pair of eye glasses.

56. The system according to claim 44, wherein said image sensor comprises:

a first optical line sensor having a scanning direction that deviates slightly from a vertical direction when said pair of eye glasses are worn; and

a second optical line sensor having a scanning direction that is substantially perpendicular to that of said first optical line sensor;

with scanning directions of said first and second optical line sensors both being substantially parallel to a frontal plane of the person when said pair of eye glasses are worn.

57. The system according to claim 44, wherein said at least one IR signal source is operable in an energy-saving mode that is interruptible by receipt of a wireless signal.

58. The system according to claim 44, wherein additional gaze points are determined for additional people relative to the picture of the region of space;

wherein said pair of eye glasses is configured to generate an eye-tracking signal and an IR signal source tracking signal for each person; and

wherein said data processing and storage unit is configured to calculate a gaze point of each person with respect to the picture acquired by a camera in said pair of eye glasses.

59. A method for determining a gaze point of a person relative to a picture of a region of space, and comprising:

(a) emitting an infrared (IR) signal from at least one IR signal source;

(b) receiving at a receiver the at least one IR signal;

(c) determining an angle of incidence of the received at least one IR signal;

(d) acquiring at a camera the picture of the region of space containing the at least one IR signal source;

(e) determining positions of the at least one IR signal source relative to the picture;

(f) determining a gaze point of the person relative to the positions of the at least one IR signal source based on information extracted from images of an eye of the person including location of a reflection on a cornea of the eye from emitted light pulses and position of a pupil of the eye; and

(g) outputting a combined image of the region of space and the gaze point.

60. The method according to claim 59, wherein the gaze points are determined for a plurality of persons relative to the picture of the region of space, and further comprising performing steps (a) and (d);

and for each person performing steps (b), (c), (e) and (f); and

wherein outputting the combined image of the region of space and the gaze points is based on the plurality of persons.