INTERFACE USING EYE TRACKING CONTACT LENSES

• US 20120281181A1
• Filed: 05/05/2011
• Published: 11/08/2012
• Est. Priority Date: 05/05/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Chen, Ruxin, Kalinli, Ozlem
• Granted Patent: US 8,632,182 B2
• Time in Patent Office: Days
• US Class Current: 351/159.03
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 30/35 using reflective optical elements in the optical path between the images and the observer
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G02C 2200/02 Magnetic means
• CPC Class Codes: G02C 7/04 Contact lenses for the eyes disinfection or sterilisation of contact lenses A61L12/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

Methods of eye gaze tracking are provided using magnetized contact lenses tracked by magnetic sensors and/or reflecting contact lenses tracked by video-based sensors. Tracking information of contact lenses from magnetic sensors and video-based sensors may be used to improve eye tracking and/or combined with other sensor data to improve accuracy. Furthermore, reflective contact lenses improve blink detection while eye gaze tracking is otherwise unimpeded by magnetized contact lenses. Additionally, contact lenses may be adapted for viewing 3D information.

Claims

1. A method of eye gaze tracking, the method comprising:

receiving, at one or more computer systems, first tracking information obtained from one or more magnetic sensors tracking a first magnetic contact lens associated with a first eye of a wearer;

receiving, at one or more computer systems, second tracking information obtained from one or more video-based sensors tracking the first magnetic contact lens using a first set of one or more reflective patterns of the first magnetic contact lens;

generating, with one or more processors associated with the one or more computer systems, augmented tracking information based on the first tracking information obtained from the one or more magnetic sensors tracking the first magnetic contact lens and the second tracking information obtained from the one or more video-based sensors tracking the first magnetic contact lens using the first set of one or more reflective patterns of the first magnetic contact lens;

determining, with the one or more processors associated with the one or more computer systems, eye gaze information based on the augmented tracking information; and

performing, with the one or more computer system, one or more actions based on the determined eye gaze information.

2. The method of claim 1 wherein generating the augmented tracking information based on the first tracking information obtained from the one or more magnetic sensors tracking the first magnetic contact lens and the second tracking information obtained from the one or more video-based sensors tracking the first magnetic contact lens using the first set of one or more reflective patterns of the first magnetic contact lens comprises:

establishing a first position of the first eye associated with the first magnetic contact lens; and

refining the first position of the first eye into a second position of the first eye.

3. The method of claim 1 further comprising:

receiving third tracking information obtained from the one or more video-based sensors tracking the head of the wearer; and

wherein generating the augmented tracking information based on the first tracking information obtained from the one or more magnetic sensors tracking the first magnetic contact lens and the second tracking information obtained from the one or more video-based sensors tracking the first magnetic contact lens using the first set of one or more reflective patterns of the first magnetic contact lens comprises determining location of the first magnetic contact lens relative to the head of the wearer.

4. The method of claim 1 further comprising:

receiving third tracking information obtained from the one or more magnetic sensors tracking a second magnetic contact lens associated with a second eye of the wearer;

receiving fourth tracking information obtained from the one or more video-based sensors tracking the second magnetic contact lens using a second set of one or more reflective patterns of the second magnetic contact lens.

5. The method of claim 4 wherein generating the augmented tracking information based on the first tracking information obtained from the one or more magnetic sensors tracking the first magnetic contact lens and the second tracking information obtained from the one or more video-based sensors tracking the first magnetic contact lens using the first set of one or more reflective patterns of the first magnetic contact lens comprises generating the augmented tracking information based on the third tracking information obtained from the one or more magnetic sensors tracking the second magnetic contact lens and the fourth tracking information obtained from the one or more video-based sensors tracking the second magnetic contact lens using the second set of one or more reflective patterns of the second magnetic contact lens.

6. The method of claim 1 wherein receiving the first tracking information obtained from the one or more magnetic sensors tracking the first magnetic contact lens or receiving the second tracking information obtained from the one or more video-based sensors tracking the first magnetic contact lens using the first set of one or more reflective patterns of the first magnetic contact lens comprises receiving at least one of a position, orientation, and movement of an eye associated with the magnetic contact lens.

7. The method of claim 1 further comprising determining blink information based on the second tracking information obtained from the one or more video-based sensors tracking the first magnetic contact lens using the first set of one or more reflective patterns of the first magnetic contact lens.

8. The method of claim 1 further comprising:

receiving third tracking information obtained from one or more magnetic sensors tracking a second magnetic contact lens associated with a first eye of a wearer of the second contact lens;

receiving fourth tracking information obtained from one or more video-based sensors tracking the second magnetic contact lens using a second set of one or more reflective patterns of the second magnetic contact lens, the second set of one or more reflective patterns of the second magnetic contact lens being different from the first set of one or more reflective patters of the first contact lens; and

distinguishing between the wearer of the first magnetic contact lens and the wearer of the second magnetic contact lens.

9. The method of claim 1 wherein performing the one or more actions based on the determined eye gaze information comprises controlling at least one of a virtual character, object, or menu of at least one of a video game, projected virtual user interface, or augmented reality user interface.

10. The method of claim 1 wherein performing the one or more actions based on the determined eye gaze information comprises controlling one or more application menus associated with a computer application.

11. An eye tracking contact lens comprising:

material forming a lens of suitable size to fit a wearer'"'"'s eye and extend over at least a portion of the sclera of the wearer'"'"'s eye, the lens having an optical zone and a peripheral zone;

one or more magnetic elements associated with the material forming the lens, the one or more magnetic elements either forming a magnetic field or configured to interfere with a magnetic field; and

one or more reflective elements associated with the material forming the lens, the one or more reflective elements configured to reflect a predetermined portion of the electromagnetic spectrum.

12. The eye tracking contact lens of claim 11 wherein the one or more magnetic elements associated with the material forming the lens comprise a plurality of magnetized nanoparticles embedded in the material forming the lens.

13. The eye tracking contact lens of claim 11 wherein the one or more magnetic elements associated with the material forming the lens comprise a plurality of magnetic elements clustered in a portion of the peripheral zone.

14. The eye tracking contact lens of claim 11 wherein the one or more reflective elements associated with the material forming the lens reflect infrared or near infrared light.

15. The eye tracking contact lens of claim 11 wherein the one or more reflective elements associated with the material forming the lens comprise a set of circular bands distributed in the peripheral zone.

16. The eye tracking contact lens of claim 11 wherein the one or more reflective elements associated with the material forming the lens comprise a set of reflective shapes distributed in the peripheral zone.

17. A contact lens for viewing 3D information, the contact lens comprising:

material forming a lens of suitable size to fit a wearer'"'"'s eye and extend over at least a portion of the sclera of the wearer'"'"'s eye, the lens having an optical zone and a peripheral zone; and

material associated with at least the optical zone that is configured to filter the 3D information to obtain a portion of the 3D information designated for the wearer'"'"'s eye.

18. The contact lens of claim 17 wherein the material associated with at least the optical zone that is configured to filter the 3D information to obtain a portion of the 3D information designated for the wearer'"'"'s eye comprises a color filter layer associated with at least one of a plurality chromatically opposite colors.

19. The contact lens of claim 17 wherein the material associated with at least the optical zone that is configured to filter the 3D information to obtain a portion of the 3D information designated for the wearer'"'"'s eye comprises a polarization filter layer associated with at least one of a plurality of light polarizations.

20. The contact lens of claim 17 wherein the material associated with at least the optical zone that is configured to filter the 3D information to obtain a portion of the 3D information designated for the wearer'"'"'s eye comprises a liquid crystal layer that obscures the optical zone to block a portion of the 3D information that is not designated for the wearer'"'"'s eye.