Extensible context-aware natural language interactions for virtual personal assistants

• US 10,127,224 B2
• Filed: 08/30/2013
• Issued: 11/13/2018
• Est. Priority Date: 08/30/2013
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Deleeuw, William C.
• Primary Examiner(s): Leland, III, Edwin S
• Application Number:
  US14/129,645
• Publication Number:
  US 20160179787A1
• Time in Patent Office: 1,901 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 16/638 Presentation of query results
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Technologies for extensible, context-aware natural language interactions include a computing device having a number of context source modules. Context source modules may be developed or installed after deployment of the computing device to a user. Each context source module includes a context capture module, a language model, one or more database query mappings, and may include one or more user interface element mappings. The context capture module interprets, generates, and stores context data. A virtual personal assistant (VPA) of the computing device indexes the language models and generates a semantic representation of a user request that associates each word of the request to a language model. The VPA translates the user request into a database query, and may generate a user interface element for the request. The VPA may execute locally on the computing device or remotely on a cloud server. Other embodiments are described and claimed.

Claims

1. A computing device for automatically querying a database for contextual natural language processing, the computing device comprising:

a plurality of context source modules;

a plurality of language models, wherein each language model is associated with a context source module of the plurality of context source modules; and

a metadata interpretation module to index the plurality of language models to determine a plurality of important words of each of the plurality of language models that are important for the corresponding context source module; and

a request interpretation module to;

determine, for each of the plurality of language models, a relevance measure of a plurality of words of a textual representation of a user request based on the plurality of important words of the corresponding language model;

generate a ranking of the determined relevance measures corresponding to the plurality of language models;

generate, based on the ranking of the determined relevance measures, a semantic representation of the textual representation of the user request;

generate a database query as a function of the semantic representation using a database query mapping of a first context source module of the plurality of context source modules, the first context source module associated with a word of the textual representation; and

apply the database query generated as a function of the semantic representation.

2. The computing device of claim 1, wherein the request interpretation module is further to:

generate a plurality of interpretations of the textual representation, each interpretation comprising a plurality of word groups selected from the words of the textual representation;

generate, for each word group of each interpretation, a ranking of each of the plurality of language models that orders the plurality of language models based on relevancy to the word group based on the plurality of important words of the corresponding language model; and

generate a composite ranking for each interpretation based on the rankings of the word groups of the associated interpretation,

wherein to generate the semantic representation comprises to generate the semantic representation as a function of the composite rankings of the interpretations.

3. The computing device of claim 2, wherein the request interpretation module is further to calculate, for each word group of each interpretation, a term frequency-inverse document frequency (TF-IDF) value, and

wherein to generate, for each word group of each interpretation, the ranking of each of the plurality of language models comprises to generate, for each word group of each interpretation, the ranking of each of the plurality of language models based on the calculated TF-IDF values.

4. The computing device of claim 1, wherein the request interpretation module is further to:

select a user interface element for generation from a plurality of user interface elements as a function of the semantic representation using a user interface element mapping of the first context source module; and

generate the selected user interface element as a function of the semantic representation, using the user interface element mapping of the first context source module.

5. The computing device of claim 4, wherein to apply the database query comprises to apply the database query to a context database to generate query results;

and

wherein the computing device further comprises a user interface module to present the query results using the user interface element.

6. The computing device of claim 1, wherein to generate the semantic representation comprises to determine a context state of the user request.

7. The computing device of claim 1, wherein the request interpretation module is further to generate a canonical representation of the user request using the semantic representation and the plurality of language models, wherein the canonical representation uses standardized vocabulary different from vocabulary in the user request.

8. The computing device of claim 1, further comprising:

a sensor input module to receive context source data;

wherein each of the plurality of language models is associated with a context source module capable of capturing the context source data.

9. A virtual personal assistant (VPA) server for automatically querying a database for contextual natural language processing, the VPA server comprising:

a speech recognition engine to (i) receive, from a computing device, audio input data representing a request spoken by a user of the computing device and (ii) produce a textual representation of the user request based on the audio input data, the textual representation including a plurality of words;

a metadata interpretation module to;

receive a plurality of language models and associated database query mappings from the computing device; and

index the plurality of language models to determine a plurality of important words of each of the plurality of language models; and

a request interpretation module to;

determine, for each of the plurality of language models, a relevance measure of a plurality of words of the textual representation of the user request based on the plurality of important words of the corresponding language model;

generate a ranking of the determined relevance measures corresponding to the plurality of language models;

generate, based on the ranking of the determined relevance measures, a semantic representation of the textual representation;

generate a database query as a function of the semantic representation using a database query mapping associated with a first language model of the plurality of language models, the first language model associated with a word of the textual representation;

apply the database query to a context database to generate query results; and

transmit the query results from the VPA server to the computing device.

10. The VPA server of claim 9, wherein the request interpretation module is further to:

generate a plurality of interpretations of the textual representation, wherein each interpretation comprises a plurality of word groups selected from the words of the textual representation;

generate, for each word group of each interpretation, a ranking of each of the plurality of language models that orders the plurality of language models based on relevancy to the word group based on the plurality of important words of the corresponding language model; and

generate a composite ranking for each interpretation based on the rankings of the word groups of the associated interpretation,

wherein to generate the semantic representation comprises to generate the semantic representation as a function of the composite rankings of the interpretations.

11. The VPA server of claim 9, wherein:

the metadata interpretation module is further to receive a plurality of user interface element mappings from the computing device, wherein each of the user interface element mappings is associated with a language model; and

the request interpretation module is further to;

select a user interface element for generation from a plurality of user interface elements as a function of the semantic representation using a user interface element mapping associated with the first language model; and

generate the selected user interface element as a function of the semantic representation, using the user interface element mapping associated with the first language model; and

transmit the user interface element from the VPA server to the computing device.

12. The VPA server of claim 9, wherein to generate the semantic representation comprises to determine a context state of the user request.

13. The VPA server of claim 9, wherein the request interpretation module is to generate a canonical representation of the user request using the semantic representation and the plurality of language models, wherein the canonical representation uses standardized vocabulary different from vocabulary in the user request.

14. One or more non-transitory computer-readable storage media device for automatically querying a database for contextual natural language processing comprising a plurality of instructions that in response to being executed cause a computing device to:

index a plurality of language models to determine a plurality of important words of each of the plurality of language models that are important for the corresponding context source module, wherein each language model is associated with a context source module of a plurality of context source modules;

determine, for each of the plurality of language models, a relevance measure of a plurality of words of a textual representation of a user request based on the plurality of important words of the corresponding language model;

generate a ranking of the determined relevance measures corresponding to the plurality of language models;

generate a semantic representation of the textual representation of the user request;

generate a database query as a function of the semantic representation using a database query mapping of a first context source module of the plurality of context source modules, the first context source module associated with a word of the textual representation; and

apply the database query generated as a function of the semantic representation.

15. The one or more non-transitory computer-readable storage media of claim 14, further comprising a plurality of instructions that in response to being executed cause the computing device to:

generate a plurality of interpretations of the textual representation, each interpretation comprising a plurality of word groups selected from the words of the textual representation;

generate, for each word group of each interpretation, a ranking of the language models that orders the language models based on relevancy to the word group; and

generate composite ranking for each interpretation based on the rankings of the word groups of the associated interpretation,

wherein to generate the semantic representation comprises to generate the semantic representation as a function of the composite rankings of the interpretations.

16. The one or more non-transitory computer-readable storage media of claim 14, further comprising a plurality of instructions that in response to being executed cause the computing device to:

select a user interface element for generation from a plurality of user interface elements as a function of the semantic representation using a user interface element mapping of the first context source module; and

generate the user interface element as a function of the semantic representation, using the user interface element mapping of the first context source module.

17. The one or more non-transitory computer-readable storage media of claim 14, wherein to index the plurality of language models comprises to index a language model after deployment of the computing device to the user.

18. One or more non-transitory computer-readable storage media for automatically querying a database for contextual natural language processing comprising a plurality of instructions that in response to being executed cause a virtual personal assistant (VPA) server to:

receive, from a computing device, audio input data representing a request spoken by a user of the computing device;

produce a textual representation of the user request based on the audio input data, the textual representation including a plurality of words;

receive a plurality of language models and associated database query mappings from the computing device;

index the plurality of language models to determine a plurality of important words of each of the plurality of language models;

determine, for each of the plurality of language models, a relevance measure of a plurality of words of the textual representation of the user request based on the plurality of important words of the corresponding language model;

generate a ranking of the determined relevance measures corresponding to the plurality of language models;

generate, based on the ranking of the determined relevance measures, a semantic representation of the textual representation;

generate a database query as a function of the semantic representation using a database query mapping associated with a first language model of the plurality of language models, the first language model associated with a word of the textual representation;

apply the database query to a context database to generate query results; and

transmit the query results from the VPA server to the computing device.

19. The one or more non-transitory computer-readable storage media of claim 18, further comprising a plurality of instructions that in response to being executed cause the computing device to:

generate a plurality of interpretations of the textual representation, each interpretation comprising a plurality of word groups selected from the words of the textual representation;

generate, for each word group of each interpretation, a ranking of each of the plurality of language models that orders the plurality of language models based on relevancy to the word group based on the plurality of important words of the corresponding language model; and

generate a composite ranking for each interpretation based on the rankings of the word groups of the associated interpretation,

wherein to generate the semantic representation comprises to generate the semantic representation as a function of the composite rankings of the interpretations.

20. The one or more non-transitory computer-readable storage media of claim 18, further comprising a plurality of instructions that in response to being executed cause the VPA server to:

receive a plurality of user interface element mappings from the computing device, wherein each of the user interface element mappings is associated with a language model;

select a user interface element for generation from a plurality of user interface elements as a function of the semantic representation using a user interface element mapping associated with the first language model; and

generate the selected user interface element as a function of the semantic representation, using the user interface element mapping associated with the first language model; and

transmit the user interface element from the VPA server to the computing device.