Directing voice input based on eye tracking

• US 9,966,079 B2
• Filed: 03/24/2014
• Issued: 05/08/2018
• Est. Priority Date: 03/24/2014
• Status: Active Grant

• Current Assignee: Lenovo PC International Limited (Lenovo Group Ltd.)
• Original Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Inventors: VanBlon, Russell Speight, Bijjaragi, Manisha, Grimme, Jason, Guy, Spencer Michael, McDaniel, Kyle Jared
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US14/223,762
• Publication Number:
  US 20150269943A1
• Time in Patent Office: 1,506 Days
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/226 using non-speech characteristics

Abstract

For directing voice input based on eye tracking, a method is disclosed that includes receiving, by use of a processor, an audio signal, the audio signal comprising voice input, selecting a location from the group consisting of a computing device and an application being displayed on a screen of the computing device, the location being detected where a user'"'"'s eyes are directed, and directing the voice input to the selected location.

Claims

1. An apparatus comprising:

a processor;

a screen accessible to the processor;

a memory that stores code executable by the processor, the code comprising;

code that receives an audio signal, the audio signal comprising voice input;

code that selects coordinates on the screen, the coordinates being detected where a user'"'"'s eyes are directed while the audio signal is received;

code that directs the voice input to an application containing the coordinates;

code that transmits the coordinates to the application;

code that performs a command corresponding to the voice input at the coordinates in the application; and

code that discards the voice input in response to the user'"'"'s eyes being directed off the screen.

2. The apparatus of claim 1, further comprising code that selects the application based on the application being detected at the coordinates on the screen.

3. The apparatus of claim 1, further comprising code that generates the command based on the voice input, the code directing the command to the coordinates.

4. The apparatus of claim 1, wherein the code employs facial recognition to recognize the face of the user.

5. A method comprising:

receiving, by use of a processor, an audio signal, the audio signal comprising voice input;

selecting coordinates on a screen of a computing device, the coordinates being detected where a user'"'"'s eyes are directed while the audio signal is received;

converting the voice input to a command;

directing the command to an application containing the coordinates;

performing the command at the coordinates in the application; and

discarding the voice input in response to failing to select an application.

6. The method of claim 5, further comprising selecting the application at the coordinates on the screen.

7. The method of claim 5, wherein selecting the coordinates comprises using facial recognition to identify the user.

8. A program product comprising a non-transitory computer readable storage medium that stores code executable by a processor to perform:

receiving, by use of a processor, an audio signal, the audio signal comprising voice input;

selecting coordinates on a screen of a computing device, the coordinates being detected where a user'"'"'s eyes are directed while the audio signal is received;

converting the voice input to a command;

directing the command to an application containing the coordinates;

performing the command at the coordinates in the application; and

discarding the voice input in response to failing to select an application.

9. The program product of claim 8, wherein the code further selects the application based on the application being detected at the coordinates on the screen.

10. The program product of claim 8, wherein directing the command comprises transmitting the audio signal to the coordinates.

11. The program product of claim 8, wherein selecting the coordinates comprises using facial recognition to identify the user.