System and Method for Using Eye Gaze Information to Enhance Interactions

• US 20140184550A1
• Filed: 03/07/2014
• Published: 07/03/2014
• Est. Priority Date: 09/07/2011
• Status: Abandoned Application

• Current Assignee: Mirametrix Inc. (Lattice Semiconductor Corporation)
• Original Assignee: TandemLaunch Technologies, LLC
• Inventors: HENNESSEY, Craig A., FISET, Jacob, ST-HILAIRE, Simon
• Application Number:
  US14/200,791
• Publication Number:
  US 20140184550A1
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means

Abstract

A system and method are provided for enhancing inputs or interactions. The method comprises correlating gaze information for a subject to information corresponding to an environment; and providing an enhancement to an input or interaction between the subject and the environment. A system and method are also provided for enabling enhanced inputs or interactions with objects in an environment. The method comprises correlating gaze information for a subject to a registration input corresponding to an object in the environment; and registering a position of the object in the environment using the gaze information.

Claims

1. A method of enhancing inputs or interactions, the method comprising:

correlating gaze information for a subject to information corresponding to an environment; and

providing an enhancement to an input or interaction between the subject and the environment.

2. The method of claim 1, wherein the information corresponding to the environment comprises a location of an object or system in the environment.

3. The method of claim 2, wherein the correlating comprises comparing a point of gaze (POG) or line of sight (LOS) intersection of the subject to the location of the object or system in the environment.

4. The method of claim 3, wherein the enhancement comprises detecting the input or interaction from the subject and applying the input or interaction to the environment according to the POG of the subject.

5. The method of claim 1, wherein the gaze information is obtained using a gaze tracking module.

6. The method of claim 1, wherein the information corresponding to the environment is obtained from metadata provided by a system, device, or entity associated with the environment.

7. The method of claim 3, wherein the input or interaction from the subject is detected using an input/interaction tracking module.

8. The method of claim 7, wherein the input/interaction tracking module is operable to obtain any one or more of an image, a video, sound, motion, and a physical interaction, from the subject.

9. The method of claim 1, wherein the enhancement comprises using the gaze information to apply a gesture to an object of interest.

10. The method of claim 9, wherein the enhancement comprises distinguishing between multiple possible objects of interest according to the gaze information.

11. The method of claim 1, wherein the enhancement comprises using the gaze information to apply a voice command to an object of interest.

12. The method of claim 9, wherein the object of interest is provided in a user interface (UI) displayed on a computer screen.

13. The method of claim 12, wherein the UI comprises any one or more of a button, a drop down selection mechanism, a scroll bar, a slider, a combo-box, a tree control, a text box, and a checkbox.

14. The method of claim 1, wherein the enhancement comprises using a first point of gaze (POG) to enable interaction with a first interface object and detection of a second POG to enable interaction with a second interface object.

15. The method of claim 14, wherein the first and second interface objects are text entry boxes.

16. The method of claim 14, wherein the first and second interface objects are application windows.

17. The method of claim 1, wherein the enhancement comprises using the gaze information to predict and perform an action on an object of interest.

18. The method of claim 1, wherein the enhancement comprises providing an input element on a touchscreen that is remote from an object of interest associated with the gaze information.

19. The method of claim 18, wherein the input element is a soft key displayed away from the object of interest.

20. The method of claim 1, wherein the enhancement comprises using the gaze information to target an object of interest on a touchscreen to enable another input to select the object of interest.

21. The method of claim 20, wherein the other input comprises any one or more of a fixation of a POG on the object of interest for a predetermined amount of time, a voice command, and a gesture.

22. The method of claim 1, wherein the enhancement comprises adjusting a sound property for at least one recipient in the environment.

23. The method of claim 22, wherein the sound property comprises a volume of the subject to be directed to the plurality of recipients.

24. The method of claim 23, wherein a plurality of recipients are communicable with the subject via a network connection.

25. The method of claim 1, wherein the enhancement comprises displaying a plurality of visual elements, each visual element associated with a different subject.

26. The method of claim 25, wherein the plurality of visual elements comprise indicators for respective points of gaze (POGs) for the corresponding subjects.

27. The method of claim 22, wherein at least two subjects are in different locations.

28. The method of claim 1, wherein the environment comprises any one or more of real-world, augmented real-world, virtual world, a two dimensional (2D) display, and a three dimensional (3D) display.

29. The method of claim 1, wherein the environment comprises an electronic sports video feed.

30. The method of claim 1, wherein the environment comprises game play.

31. The method of claim 1, wherein the environment is being viewed using a heads up interface.

37. A computer readable storage medium comprising computer executable instructions for performing the method of claim 1.

38. An electronic device comprising a processor and memory, the memory comprising computer executable instructions for causing the processor to perform the method of claim 1.

39. A tracking system comprising the electronic device of claim 38.

32. A method of enabling enhanced inputs or interactions with objects in an environment, the method comprising:

correlating gaze information for a subject to a registration input corresponding to an object in the environment; and

registering a position of the object in the environment using the gaze information.

33. The method of claim 32, further comprising obtaining an identifier for the object and associating the identifier with the object for subsequent interactions with the object.

34. The method of claim 32, wherein the gaze information comprises a plurality of points of gaze (POG) of the subject and the registering comprises defining a bounding area or volume surrounding the object according to the plurality of POGs.

35. The method of claim 32, further comprising obtaining a timestamp for a current location of the object in the environment, and updating an object position over time using new current locations to repeatedly determine a positioning of the object in the environment.

36. The method of claim 32, further comprising obtaining a range of available actions for the object in the environment, to enable subsequent interactions with the object in the environment.