Replacing pronouns with focus-specific objects in search queries

• US 10,262,036 B2
• Filed: 12/29/2016
• Issued: 04/16/2019
• Est. Priority Date: 12/29/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Paine, Llewyn Elise, Gordon, John C., Wu, Rowen
• Primary Examiner(s): Chow, Van N
• Application Number:
  US15/394,682
• Publication Number:
  US 20180189354A1
• Time in Patent Office: 838 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/24569 Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A computing device includes a logic machine and a storage machine holding instructions. The instructions are executable by the logic machine to, receive a machine-understandable representation of voice input, determine a pronoun associated with the voice input, map the pronoun to an object of user focus, generate one or more focus-specific search queries for the object of user focus, execute the one or more focus-specific search queries, and present the results of the one or more focus-specific search queries.

Claims

1. A computing device, comprising:

a logic machine; and

a storage machine holding instructions executable by the logic machine to;

receive a machine-understandable representation of voice input;

identify a pronoun and a part-of-speech other than the pronoun associated with the voice input;

map the pronoun to an object of user focus based at least on the pronoun and the part-of-speech identified in the voice input;

generate one or more focus-specific search queries for the object of user focus based at least on the part-of-speech identified in the voice input;

execute the one or more focus-specific search queries; and

present the results of the one or more focus-specific search queries.

2. The computing device of claim 1, wherein the instructions are further executable by the logic machine to:

determine a user'"'"'s gaze target; and

wherein the object of user focus is determined based on the user'"'"'s gaze target.

3. The computing device of claim 2, wherein the user'"'"'s gaze target intersects a virtual object visually presented via a display, and wherein the virtual object is determined to be the object of user focus.

4. The computing device of claim 2, wherein the user'"'"'s gaze target intersects a real-world object in a physical space, and wherein the real-world object is determined to be the object of user focus.

5. The computing device of claim 1, wherein the instructions are further executable by the logic machine to:

identify a gesture performed by a user; and

wherein the object of user focus is determined based on the gesture performed by the user.

6. The computing device of claim 1, wherein one or both of the mapping and generating are further based on a context parameter.

7. The computing device of claim 6, wherein the context parameter includes a location of the computing device.

8. The computing device of claim 6, wherein the context parameter includes an application executed by the computing device that has a user'"'"'s focus.

9. The computing device of claim 8, wherein the application is configured to visually present a graphical user interface, and wherein the instructions are further executable by the logic machine to:

present the results of the one or more focus-specific search queries as an overlay that at least partially overlaps the graphical user interface.

10. A method of focus-specific search querying performed by a computing device, comprising:

receiving a machine-understandable representation of voice input;

identifying a pronoun and at least a part-of-speech other than the pronoun associated with the voice input;

mapping the pronoun to an object of user focus based at least on the pronoun and the part-of-speech identified in the voice input;

generating one or more focus-specific search queries for the object of user focus based at least on the part-of-speech identified in the voice input;

executing the one or more focus-specific search queries; and

presenting the results of the one or more focus-specific search queries.

11. The method of claim 10, further comprising:

determining a user'"'"'s gaze target; and

wherein the object of user focus is determined based on the user'"'"'s gaze target.

12. The method of claim 10, further comprising:

identifying a gesture performed by a user; and

wherein the object of user focus is determined based on the gesture performed by the user.

13. The method of claim 10, wherein one or both of the mapping and generating are further based on a context parameter.

14. The method of claim 13, wherein the context parameter includes at least one of a location of the computing device, and an application executed by the computing device and having a user'"'"'s focus.

15. A computing device, comprising:

a near-eye display;

a gaze detection machine configured to determine a wearer'"'"'s gaze target on the near-eye display;

a microphone configured to receive voice input of the wearer;

a logic machine; and

a storage machine holding instructions executable by the logic machine to;

receive, via the microphone, voice input;

translate the voice input into a machine-understandable representation of the voice input;

identify a pronoun and a part-of-speech other than the pronoun associated with the voice input;

determine, via the gaze detection machine, an object of wearer focus based on the wearer'"'"'s gaze target;

map the pronoun to the object of wearer focus;

generate one or more focus-specific search queries for the object of wearer focus based at least on the part-of-speech identified in the voice input;

execute the one or more focus-specific search queries; and

present the results of the one or more focus-specific search queries.

16. The computing device of claim 15, wherein the near-eye display is opaque, and wherein the object of wearer focus is a virtual object visually presented via the near-eye display.

17. The computing device of claim 15,

wherein the near-eye display is at least partially see-through;

wherein the computing system further comprises an optical system configured to image a physical space; and

wherein the object of wearer focus is a real-world object in the physical space that intersects the wearer'"'"'s gaze target.

18. The computing device of claim 15,

wherein the near-eye display is at least partially see-through;

wherein the computing device further comprises;

an optical system configured to image a physical space;

a gesture recognition machine configured to identify a gesture performed by the wearer based on one or more images provided by the optical system; and

wherein the instructions are further executable by the logic machine to;

identify, via the gesture recognition machine, a gesture performed by the wearer; and

identify the object of wearer focus as a real-world object in the physical space from one or more images provided by the optical system based on the identified gesture.

19. The computing device of claim 15, wherein one or both of the mapping and generating are further based on a context parameter including at least one of a location of the computing device and an application executed by the computing device that has the wearer'"'"'s focus.