Systems and methods for changing behavior of computer program elements based on gaze input

• US 10,430,150 B2
• Filed: 08/25/2014
• Issued: 10/01/2019
• Est. Priority Date: 08/23/2013
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Lindh, Fredrik, Vennstrom, Anders, Olsson, Anders
• Primary Examiner(s): Pillai, Namitha
• Application Number:
  US14/467,877
• Publication Number:
  US 20150058812A1
• Time in Patent Office: 1,863 Days
• CPC Class Codes: G02B 27/00 Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/13 Aspects of volume control, not necessarily automatic, in stereophonic sound systems
• CPC Class Codes: H04S 2420/07 Synergistic effects of band splitting and sub-band processing
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation
• CPC Class Codes: H04S 7/307 Frequency adjustment, e.g. tone control H04S7/301 takes precedence
• CPC Class Codes: H04S 7/308 Electronic adaptation dependent on speaker or headphone connection

Abstract

According to the invention, a method for changing the behavior of computer program elements is disclosed. The method may include determining, with an eye tracking device, a gaze point of a user. The method may also include causing, with a computer system, an interactive event controlled by the computer system to alter its behavior based at least in part on the gaze point of the user.

Claims

1. A method for changing behavior of computer program elements, wherein the method includes operations comprising:

displaying content on a display device;

defining a plurality of locations in the content where artificial intelligence characters can appear;

determining, with an eye tracking device, a gaze point of a user on the display device;

determining, with the eye tracking device, which of the plurality of locations have been gazed at by the user;

causing, with a computer system, artificial intelligence characters to appear at random or in a predefined pattern at at least some of the plurality of locations;

causing, with the computer system, a character controlled by the user to select a virtual weapon based at least in part on the gaze point of the user;

receiving, with a non-gaze input device, a non-gaze input;

causing, with the computer system, an alteration in the random or predefined pattern of where artificial intelligence characters appear, the alteration based at least in part on which of the plurality of locations have been gazed at by the user; and

causing, with the computer system, the selected virtual weapon controlled by the computer system to fire toward a virtual point displayed in the content on the display device based at least in part on receipt of the non-gaze input and the gaze point of the user on the display device correlating to the virtual point.

2. The method for changing behavior of computer program elements of claim 1, wherein the virtual point comprises:

a graphical representation of a thing, creature, or other character.

3. The method for changing behavior of computer program elements of claim 1, wherein selecting the virtual weapon based at least in part on the gaze point of the user being to the left or the right of the display device further includes:

causing the character controlled by the user to switch the selected virtual weapon to an additional virtual weapon.

4. The method of claim 1, the operations further comprising:

determining, with the eye tracking device, that the gaze point of the user is to the left or the right of the display device,

wherein selection of the virtual weapon is further based at least in part on the gaze point of the user being to the left or the right of the display device.