PROVIDING SUGGESTED VOICE-BASED ACTION QUERIES

• US 20160350304A1
• Filed: 07/24/2015
• Published: 12/01/2016
• Est. Priority Date: 05/27/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Aggarwal, Vikram, Gupta, Pravir Kumar
• Granted Patent: US 10,504,509 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/3323 using document space presentation or visualization, e.g. category, hierarchy or range presentation and selection
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Technology of the disclosure may facilitate user discovery of various voice-based action queries that can be spoken to initiate computer-based actions, such as voice-based action queries that can be provided as spoken input to a computing device to initiate computer-based actions that are particularized to content being viewed or otherwise consumed by the user on the computing device. Some implementations are generally directed to determining, in view of content recently viewed by a user on a computing device, at least one suggested voice-based action query for presentation via the computing device. Some implementations are additionally or alternatively generally directed to receiving at least one suggested voice-based action query at a computing device and providing the suggested voice-based action query as a suggestion in response to input to initiate providing of a voice-based query via the computing device.

Claims

1. A computer-implemented method, comprising:

receiving, from a computing device, an indication of content recently viewed on the computing device;

determining, based on the indication of the content, an entity referenced in the content;

determining a computer-based action mapped to the entity in one or more electronic databases;

generating a suggested voice-based action query that includes at least one action term that initiates performance of the computer-based action and that includes at least one entity term selected based on the entity;

receiving, from the computing device, a voice-based query input indication, the voice-based query input indication indicating receipt of input of the user via the computing device to initiate providing of a voice-based query via the computing device; and

providing the suggested voice-based action query to the computing device in response to receiving the voice-based query input indication, the suggested voice-based action query provided to the computing device for display as a suggestion for the voice-based query.

2. The method of claim 1, further comprising:

determining an indication of a need for suggested voice-based action queries; and

providing the suggested voice-based action query for display as the suggestion based on determining the indication of the need for suggested voice-based action queries.

3. The method of claim 2, wherein determining the indication of the need for suggested voice-based action queries comprises:

determining the indication of the need for suggested voice-based action queries based on not receiving spoken input from the user at the computing device within a threshold amount of time from the input of the user to initiate providing of the voice-based query.

4. The method of claim 2, wherein determining the indication of the need for suggested voice-based action queries comprises:

determining the indication of the need for suggested voice-based action queries based on detecting at least a threshold noise level by the computing device following the input of the user to initiate providing of the voice-based query.

5. The method of claim 1, wherein receiving the indication of the content comprises receiving terms of the content and display properties for each of the terms in the content;

and wherein determining the entity referenced in the content comprises determining the entity based on at least one term of the terms and the display properties for the term in the content.

6. The method of claim 1, wherein determining the computer-based action mapped to the entity comprises determining a class of the entity, and identifying a mapping of the computer-based action to the class.

7. The method of claim 1, wherein generating the suggested voice-based action query comprises determining a generic descriptor mapped to the entity, and using the generic descriptor as the at least one entity term.

8. The method of claim 7, wherein the generic descriptor is a pronoun mapped to the entity.

9. The method of claim 7, further comprising:

identifying an image of the entity; and

providing the image of the entity to the computing device for display as an annotation for the suggested voice-based action query.

10. The method of claim 1, further comprising:

receiving, from the computing device, an indication of a preferred language of the computing device;

wherein generating the suggested voice-based action query comprises selecting the action term based on a mapping of the action term to the computer-based action and based on a mapping of the action term to the preferred language.

11. The method of claim 10, wherein generating the suggested voice-based action query comprises selecting the entity term based on the preferred language.

12. The method of claim 1, further comprising:

receiving application information from the computing device, the application information indicating at least one of;

an application installed on the computing device, and a version of the application installed on the computing device;

wherein determining the computer-based action comprises selecting the computer-based action based on a mapping of the computer-based action to at least one of;

the application installed on the computing device, and the version of the application installed on the computing device.

13. The method of claim 1, further comprising:

receiving an indication of an application in which the content was generated;

wherein determining the computer-based action comprises selecting the computer-based action based on the indication of the application in which the content was generated.

14. The method of claim 1, wherein selection of the suggested voice-based action query at the computing device causes a first application of the computing device to perform at least one aspect of the computer-based action and further comprising:

determining an additional computer-based action mapped to the entity in the at least one electronic database;

generating an additional suggested voice-based action query that includes at least one additional action term that initiates performance of the additional computer-based action and that includes the at least one entity term; and

providing the additional suggested voice-based action query to the computing device in response to receiving the voice-based query indication, the additional suggested voice-based action query provided to the computing device for display as an additional suggestion for the voice-based query;

wherein selection of the additional suggested voice-based action query at the computing device causes a second application of the computing device to perform at least one aspect of the additional computer-based action.

15. The method of claim 1, wherein the content recently viewed on the computing device is the content most recently viewed relative to the input of the user via the computing device to initiate the providing of the voice-based query via the computing device.

16. The method of claim 15, wherein the indication of the content is provided by the computing device in response to the input of the user via the computing device to initiate the providing of the voice-based query via the computing device.

17. The method of claim 1, wherein the content recently viewed on the computing device is the content most recently viewed in an application of the computing device.

18. The method of claim 17, wherein the content most recently viewed in the application of the computing device is the content currently displayed by the computing device.

19. A method, comprising:

receiving, at a computing device, input to initiate providing of a voice-based query via the computing device;

identifying, by the computing device, content displayed by the computing device most recently relative to receiving the input;

providing an indication of the content;

receiving a suggested voice-based action query that is based on the indication of the content and that includes at least one action term and at least one entity term, the entity term referencing an entity of the content and the action term being mapped to the entity and initiating performance of a computer-based action;

determining that no spoken input has been received from the user within a threshold amount of time relative to receiving the input; and

in response to determining the no spoken input has been received within the threshold amount of time, providing the suggested voice-based action query as a suggestion for the voice-based query.

20. The method of claim 19, further comprising:

providing an indication of a preferred language of the computing device with the indication of the content;

wherein the suggested voice-based action query is received in the preferred language of the computing device in response to providing the indication of the preferred language.

21. The method of claim 19, further comprising:

identifying a selection of the suggested voice-based action query; and

performing the computer-based action based on identifying the selection of the suggested voice-based action query.

22. A system, comprising:

a client device comprising a microphone, a display, and a processor executing instructions to;

monitor for spoken input via the microphone in response to receiving certain user-initiated input;

identify content displayed by the computing device most recently relative to receiving the certain user-initiated input;

provide an indication of the content;

receive a suggested voice-based action query that is based on the indication of the content and that includes at least one action term and at least one entity term, the entity term referencing an entity of the content and the action term being mapped to the entity and initiating performance of a computer-based action;

determine that no spoken input has been received from the user via the microphone within a threshold amount of time relative to receiving the certain user-initiated input; and

in response to determining the no spoken input has been received within the threshold amount of time, providing the suggested voice-based action query as a suggestion for the voice-based query.

23. The system of claim 22, further comprising:

a server in communication with the client device, the server comprising a server processor executing instructions to;

receive, from the client device, the indication of the content;

determine, based on the indication of the content, an entity referenced in the content;

determine a computer-based action mapped to the entity in one or more electronic databases;

generate the suggested voice-based action query; and

provide the suggested voice-based action query to the client device.