Customized query-action mappings for an offline grammar model

• US 9,836,527 B2
• Filed: 02/24/2016
• Issued: 12/05/2017
• Est. Priority Date: 02/24/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Gao, Yuli, Sung, Sangsoo, Mengibar, Pedro Jose Moreno
• Primary Examiner(s): SPOONER, LAMONT M
• Application Number:
  US15/051,778
• Publication Number:
  US 20170242914A1
• Time in Patent Office: 650 Days
• Field of Search: 704 1, 704 9, 704 10, 704251, 704257
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

An offline semantic processor of a resource-constrained voice-enabled device such as a mobile device utilizes an offline grammar model with reduced resource requirements to parse voice-based queries. In various implementations, a query issued at a resource-constrained device may be semantically processed to identify candidate responsive actions that are performable by the resource-constrained device. Candidate responsive action performance statistics may be analyzed to select, from the one or more candidate responsive actions, a qualifying responsive action. In various implementations, the candidate responsive action performance statistics may relate to performance of the one or more candidate responsive actions by the resource-constrained device following issuance of the query. An offline grammar model stored in the resource-constrained device may be updated to include a mapping between the issued query and the qualifying responsive action, wherein the offline grammar model maps queries to actions that are performable by the resource-constrained device while offline.

Claims

1. A method, comprising:

semantically processing, by an online semantic processor, a long tail voice-based query issued at a first resource-constrained device of a coordinated ecosystem of resource-constrained devices controlled by a user to identify one or more candidate responsive actions that are performable by one or more of the resource-constrained devices, wherein the long tail voice-based query is unique or occurs relatively infrequently in a search volume of queries;

providing, as output at one or more of the resource-constrained devices, one or more prompts suggesting performance of the respective one or more candidate actions;

analyzing one or more responses by the user to the one or more prompts to select, from the one or more candidate responsive actions, a qualifying responsive action; and

updating an offline grammar model stored in each of the coordinated ecosystem of resource-constrained devices to include a mapping between the issued long tail voice-based query and the qualifying responsive action, wherein the offline grammar model maps voice-based queries to actions that are performable by each resource-constrained device of the coordinated ecosystem of resource-constrained devices while offline.

2. The method of claim 1, wherein the one or more prompts comprise a selectable graphical element.

3. The method of claim 1, wherein the one or more prompts comprise an audible or tactile prompt.

4. The method of claim 1, further comprising communicating the offline grammar model to each resource-constrained device of the coordinated ecosystem of resource-constrained devices for storage and for use by respective semantic processors of each resource-constrained device of the coordinated ecosystem of resource-constrained devices while offline.

5. The method of claim 4, wherein the communicating dynamically updates the offline grammar model stored in the resource-constrained device.

6. The method of claim 1, wherein the semantic processing is performed using an online grammar model, wherein the offline grammar model has reduced resource requirements relative to the online grammar model.

7. The method of claim 1, further comprising restricting access to the one or more of the mapping, the issued query, and the offline grammar to the resource-constrained device.

8. A system including memory and one or more processors operable to execute instructions stored in the memory, comprising instructions to:

semantically process a long tail voice-based query issued at a first resource-constrained device of a coordinated ecosystem of resource-constrained devices controlled by a user to identify one or more candidate responsive actions that are performable by one or more of the resource-constrained devices, wherein the long tail voice-based query is unique or occurs relatively infrequently in a search volume of queries;

provide, as output at one or more of the resource-constrained devices, one or more prompts suggesting performance of the respective one or more candidate actions;

analyze one or more responses by the user to the one or more prompts to select, from the one or more candidate responsive actions, a qualifying responsive action; and

update an offline grammar model stored in each of the coordinated ecosystem of resource-constrained devices to include a mapping between the issued long tail voice-based query and the qualifying responsive action, wherein the offline grammar model maps voice-based queries to actions that are performable by each resource-constrained device of the coordinated ecosystem of resource-constrained devices while offline.

9. The system of claim 8, wherein the one or more prompts comprise a selectable graphical element.

10. The system of claim 8, wherein the one or more prompts comprise an audible or tactile prompt.

11. The system of claim 8, further comprising instructions to communicate the offline grammar model to each resource-constrained device of the coordinated ecosystem of resource-constrained devices for storage and for use by respective semantic processors of each resource-constrained device of the coordinated ecosystem of resource-constrained devices while offline.

12. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by a computer system, cause the computer system to perform the following operations:

semantically processing, by an online semantic processor, a long tail voice-based query issued at a first resource-constrained device of a coordinated ecosystem of resource-constrained devices controlled by a user to identify one or more candidate responsive actions that are performable by one or more of the resource-constrained devices, wherein the long tail voice-based query is unique or occurs relatively infrequently in a search volume of queries;

providing, as output at one or more of the resource-constrained devices, one or more prompts suggesting performance of the respective one or more candidate actions;

analyzing one or more responses by the user to the one or more prompts to select, from the one or more candidate responsive actions, a qualifying responsive action; and

updating an offline grammar model stored in each of the coordinated ecosystem of resource-constrained devices to include a mapping between the issued long tail voice-based query and the qualifying responsive action, wherein the offline grammar model maps voice-based queries to actions that are performable by each resource-constrained device of the coordinated ecosystem of resource-constrained devices while offline.