Eye Tracking System

• US 20180074581A1
• Filed: 03/23/2016
• Published: 03/15/2018
• Est. Priority Date: 03/23/2015
• Status: Active Grant

• Current Assignee: Controlrad Systems Inc.
• Original Assignee: Controlrad Systems Inc.
• Inventors: Melman, Haim, Eden, Nir
• Granted Patent: US 10,761,601 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A system for evaluating a point of gaze of a user on an object, the system comprising: an eye tracker, comprising: at least one eye tracker camera associated with a first coordinate system; a light source; at least one computer; at least one positioning camera associated with a second coordinate system, the geometrical data of the object is stored in a storage device accessible to the at least one computer; wherein any combination of the at least one computer is configured to determine: a gaze line of the user relative to the first coordinate system; the position of the at least one eye tracker camera relative to the second coordinate system; the position of the object relative to the second coordinate system; and a gaze point, based on the gaze line in any coordinate system and the at least one object'"'"'s position in the any coordinate system.

Claims

1-31. -31. (canceled)

32. A system for evaluating a point of gaze of a user on an object, said system comprising:

at least one eye tracker camera and at least one positioning camera, said at least one eye tracker camera and said at least one positioning camera are configured to be arranged in fixed positions relative to each other;

at least one light source configured to provide corneal reflection from at least one of said user'"'"'s eyes;

at least one computer; and

at least one object, wherein the geometrical data of said at least one object is stored in a storage device accessible to said at least one computer;

wherein any combination of said at least one computer is configured to determine;

(a) a gaze line of said user relative to a 3D coordinate system, based on at least one image received from said at least one eye tracker camera;

(b) the position of said at least one object relative to said 3D coordinate system, based on an image received from said at least one positioning camera; and

(c) a gaze point calculated as the intersection of said gaze line with the surface of said object in said 3D coordinate system, based on said gaze line, said at least one object position and said at least one object geometrical data.

33. The system of claim 32, wherein said at least one object comprises at least two objects;

said at least two objects are configured to be arranged in fixed positions relative to each other.

34. The system of claim 32, wherein said at least one object is a monitor, said monitor is configured to display at least one marker.

35. The system of claim 32, wherein said at least one positioning camera comprises at least two positioning cameras;

said at least two positioning cameras are configured to enable determining the position of said at least one object using stereoscopic methods.

36. A system for evaluating a point of gaze of a user on an object, said system comprising:

at least one eye tracker camera associated with a first 3D coordinate system that is fixed relative to the at least one eye tracker camera;

at least one light source configured to provide corneal reflection from at least one of said user'"'"'s eyes;

at least one computer;

at least one object, wherein the geometrical data of said at least one object is stored in a storage device accessible to said at least one computer; and

at least one positioning camera configured to be arranged in a fixed position relative to said at least one object;

said at least one positioning camera configured to be associated with a second 3D coordinate system that is fixed relative to said at least one positioning camera;

wherein any combination of said at least one computer is configured to determine;

(a) a gaze line of said user relative to said first 3D coordinate system, based on at least one image received from said at least one eye tracker camera;

(b) the position of said at least one eye tracker camera relative to said second 3D coordinate system, thereby determining the position of one 3D coordinate system in the other 3D coordinate system, based on at least one image received from said at least one positioning camera; and

(c) a gaze point calculated as the intersection of said gaze line with the surface of said object in any 3D coordinate system, based on said gaze line in said any 3D coordinate system, said at least one object'"'"'s position in said any 3D coordinate system and said at least one object'"'"'s geometrical data.

37. The system of claim 36, wherein said at least one object comprises at least two objects;

wherein said at least two objects are configured to be arranged in fixed positions relative to each other.

38. The system of claim 36, wherein said at least one positioning camera comprises at least two positioning cameras;

said at least two positioning cameras are configured to enable determining the position of said at least one object using stereoscopic methods.

39. A system for evaluating a point of gaze of a user on an object, said system comprising:

an eye tracker, comprising;

at least one eye tracker camera associated with a first 3D coordinate system that is fixed relative to said at least one eye tracker camera;

a light source configured to provide corneal reflection from at least one of said user'"'"'s eyes; and

at least one computer;

at least one positioning camera associated with a second 3D coordinate system that is fixed relative to said at least one positioning camera; and

at least one object, wherein the geometrical data of said at least one object is stored in a storage device accessible to the at least one computer;

wherein any combination of said at least one computer is configured to determine;

(a) a gaze line of said user relative to said first 3D coordinate system, based on an image received from said at least one eye tracker camera;

(b) the position of said at least one eye tracker camera relative to said second 3D coordinate system, thereby determining the position of one 3D coordinate system in the other 3D coordinate system, based on an image received from said at least one positioning camera;

(c) the position of said at least one object relative to said second 3D coordinate system based on said image from said at least one positioning camera and said stored geometrical data of said at least one object; and

(d) a gaze point calculated as the intersection of said gaze line with the surface of said object in any 3D coordinate system, based on said gaze line in said any 3D coordinate system and said at least one object'"'"'s position in said any 3D coordinate system.

40. The system of claim 39, wherein a first positioning camera is configured to determine the position of said at least one eye tracker, said first positioning camera associated with a second coordinate system fixed relative to said first positioning camera;

wherein a second positioning camera is configured to determine the position of said at least one object, said second positioning camera associated with a third coordinate system fixed relative to said second positioning camera; and

wherein the intersection of a gaze line determined by said at least one eye tracker with said at least one object is determined based on a transformation between said second coordinate system and said third coordinate system.

41. The system of claim 39, wherein said at least one object comprises at least two objects;

wherein said at least two objects are configured to be arranged in fixed positions relative to each other.

42. The system of claim 39, wherein said at least one object is a monitor, said monitor is configured to display at least one marker.

43. The system of claim 39, wherein said at least one positioning camera comprises at least two positioning cameras;

said at least two positioning cameras are configured to enable determining the position of said at least one object using stereoscopic methods.

44. A method of calibrating and using an eye tracker configured to track a gaze point of a user on a display, wherein said display is configured to be positioned in at least one position different from a calibration position relative to said eye tracker for said tracking, said method comprising the steps of:

placing said display in a position relative to said eye tracker so that said display covers the entire area covered by the gaze range of said eye tracker;

determining the position of said display and said eye tracker relative to each other;

displaying a plurality of targets on said display, each of said plurality of targets provides calibration parameters of a gaze line for this target in a first coordinate system, based on the relative position of said eye tracker and said display, wherein said plurality of targets cover the area covered by said gaze range of said eye tracker;

storing said calibration data;

positioning at least one of said display and said eye tracker in a usage position that is different from said calibration position relative to each other;

employing at least one positioning camera thereby determining the position of said display and said eye tracker relative to each other;

determining a gaze line in said first coordinate system based on said calibration data; and

determining an intersection of said gaze line with said display in any coordinate system based on the relative position of said display and said eye tracker as determined based on at least one image from said positioning camera, thereby providing said gaze point on said display.

45. The method of claim 44, wherein determining the position of said display and said eye tracker relative to each other is performed using a positioning camera.

46. A method of calibrating an eye tracker for use with a display, said eye tracker and said display are configured to be positioned relative to each other in at least two different positions for tracking a gaze point on said display, said method comprising the steps of:

placing said display in a first position thereby covering at least a part of the angles available by the gaze range of said eye tracker;

determining the position of said display and said eye tracker relative to each other;

displaying a plurality of targets on said display, each of said plurality of targets provides calibration parameters of a gaze line for this target in a first coordinate system, based on the relative position of said eye tracker and said display, wherein said plurality of targets cover at least a part of the area of said display;

repeating at least once, the above steps for another position of said display thereby calibrating additional gaze lines available by said eye tracker'"'"'s gaze range;

and

storing said calibration data.

47. The method of claim 46, wherein determining the position of said display and said eye tracker relative to each other is performed using a positioning camera.

48. A method for calibrating an eye tracker, said method comprising the steps of:

drawing the attention of a user to a calibration target;

employing at least one positioning camera and at least one computer thereby determining the position of said calibration target and said eye tracker coordinate system relative to each other;

determining the calibration parameters of a gaze line, to said calibration target, in a first coordinate system;

repeating the above steps thereby providing a desired part of a gaze range and calibration targets distribution; and

storing said calibration data.

49. The method of claim 48, wherein said calibration target comprises at least one LED.

50. The method of claim 49, wherein drawing the attention of said user comprises turning ON at least one of said at least one LED of a desired target and turning OFF the other LEDs of other targets.

51. The method of claim 49, wherein a target is distinguished by any of ON/OFF sequence pattern, color and shape defined by its at least one LED.

52. The method of claim 51, wherein said shape of a target is determined by the arrangement of said at least one LED constituting said target, the arrangement comprises any of spatial distribution, ON/OFF sequence and color.

53. The method of claim 48, wherein said calibration target comprises items.

54. The method of claim 53, wherein drawing the attention of said user comprises illuminating an item of a desired target.

55. The method of claim 53, wherein targets are distinguished by any of ON/OFF sequence pattern of illumination, color and shape.

56. The method of claim 48, wherein said target is projected on a substrate using at least one projector.

57. The method of claim 53, wherein drawing the attention of said user comprises projecting a desired target.

58. The method of claim 57, wherein targets are distinguished by any of ON/OFF sequence pattern of the projector, image color and image shape.

59. The method of claim 48, wherein after said calibration, said eye tracker position is changed so that at least one target falls in a position, in said gaze range, that did not have a target during said calibration;

said method further comprises;

repeating said calibration process using said at least one target.

60. The method of claim 48, wherein said at least one positioning camera is configured to capture at least a part of said targets.

61. The method of claim 48, wherein a computer is configured to determine the position of at least one target that is not captured by said positioning camera, and said eye tracker coordinate system relative to each other, based on at least one image received from said at least one positioning camera and the data of the spatial arrangement of at least a part of the targets that includes at least a part of the targets captured by said at least one positioning camera and said at least one target that is not captured by said at least one positioning camera.