Self-learning statistical natural language processing for automatic production of virtual personal assistants

• US 9,772,994 B2
• Filed: 07/25/2014
• Issued: 09/26/2017
• Est. Priority Date: 07/25/2013
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Karov, Yael, Breakstone, Micha, Shilon, Reshef, Keller, Orgad, Shellef, Eric
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US14/341,013
• Publication Number:
  US 20150032443A1
• Time in Patent Office: 1,159 Days
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation

Abstract

Technologies for natural language request processing include a computing device having a semantic compiler to generate a semantic model based on a corpus of sample requests. The semantic compiler may generate the semantic model by extracting contextual semantic features or processing ontologies. The computing device generates a semantic representation of a natural language request by generating a lattice of candidate alternative representations, assigning a composite weight to each candidate, and finding the best route through the lattice. The composite weight may include semantic weights, phonetic weights, and/or linguistic weights. The semantic representation identifies a user intent and slots associated with the natural language request. The computing device may perform one or more dialog interactions based on the semantic request, including generating a request for additional information or suggesting additional user intents. The computing device may support automated analysis and tuning to improve request processing. Other embodiments are described and claimed.

Claims

1. A computing device for interpreting natural language requests, the computing device comprising:

a semantic compiler module to generate a semantic model as a function of a corpus of predefined requests, wherein the semantic model includes a plurality of mappings between a natural language request and a semantic representation of the natural language request, wherein the semantic representation identifies a user intent and zero or more slots associated with the user intent; and

a request decoder module to;

(i) generate, using the semantic model, a lattice of candidate alternatives indicative of a natural language request, wherein each candidate alternative represents a token of the natural language request, and wherein the lattice comprises a plurality of routes, wherein each route represents the natural language request;

(ii) assign a composite confidence weight to each candidate alternative as a function of the semantic model;

(iii) determine an optimal route through the candidate alternative lattice from the plurality of routes based on the composite confidence weight; and

(iv) generate a semantic representation of the natural language request as a function of the candidate alternatives of the optimal route;

wherein to generate the semantic model comprises to;

(i) identify a contextual semantic feature in the corpus using a semi-supervised algorithm, wherein the contextual semantic feature comprises a sequence of lexical sets associated with a user intent and zero or more slots associated with the user intent;

(ii) determine a first probability of the contextual semantic feature given the user intent; and

(iii) determine a normalized probability of the user intent as a function of a rate of occurrence of the contextual semantic feature in the corpus; and

wherein to identify the contextual semantic feature using the semi-supervised algorithm comprises to;

(i) tag a first group of predefined sample queries in the corpus to identify user intents and slots;

(ii) extract the contextual semantic feature from the sample queries;

(iii) analyze, using the semantic model, a second group of predefined sample queries in the corpus;

(iv) extract additional contextual semantic features in response to analyzing the second group of predefined sample queries; and

(v) update the first probability and the normalized probability in response to analyzing the second group of predefined sample queries.

2. The computing device of claim 1, wherein to identify the contextual semantic feature comprises to identify the contextual semantic feature using an unsupervised algorithm, wherein to identify the contextual semantic feature using the unsupervised algorithm comprises to:

identify predefined named entities and relationships in a first group of predefined sample queries in the corpus;

cluster the predefined sample queries using an unsupervised clustering algorithm to generate a plurality of clusters; and

assign a user intent and slots to each cluster of the plurality of clusters.

3. The computing device of claim 1, wherein to generate the semantic model comprises to:

generate an ontological index as a function of a predefined ontology associated with the user intent, wherein the predefined ontology includes a plurality of objects describing a knowledge domain.

4. The computing device of claim 3, wherein to generate the ontological index comprises to:

assign a vector space to an object type of the predefined ontology, wherein the vector space includes a plurality of coordinates, wherein each coordinate encodes a lexical token representing an associated object of the predefined ontology; and

map a slot of the user intent associated with the predefined ontology to the vector space.

5. The computing device of claim 1, wherein the request decoder module is further to:

receive a representation of speech data indicative of the natural language request; and

convert the representation of speech data to a first lattice of candidate alternatives indicative of the natural language request, wherein to convert the representation of speech data comprises to convert the representation of speech data using a language model generated as a function of a domain-biased web corpus;

wherein to generate the lattice of candidate alternatives comprises to generate the lattice of candidate alternatives in response to conversion of the representation of speech data to the first lattice of candidate alternatives.

6. The computing device of claim 1, wherein to generate the lattice comprises to:

generate a candidate alternative corresponding to a user intent and associated slots of a mapping of the semantic model; and

generate a candidate alternative using a language model, as a function of phonetic similarity to the natural language request.

7. The computing device of claim 1, wherein to assign the composite confidence weight further comprises to assign a confidence weight as a function of a language model.

8. The computing device of claim 7, wherein to assign the confidence weight as a function of the language model comprises to assign a phonetic similarity score, a general language model confidence score, a domain language model score, or a syntactic confidence score.

9. The computing device of claim 1, further comprising a dialog management module to:

determine whether the semantic representation of the natural language request includes sufficient information to perform a user intent of the semantic representation;

perform the user intent in response to a determination that the semantic representation includes sufficient information;

generate a response as a function of the semantic representation;

generate a natural language representation of the response using the semantic model; and

record a user dialog session including the natural language request and the natural language representation of the response into a corpus of recorded user dialog sessions.

10. The computing device of claim 1, further comprising a tuning module to update the semantic model in response to generating the semantic representation.

11. The computing device of claim 10, wherein to update the semantic model comprises to:

determine the semantic representation was generated with no ambiguities;

identify a token of the natural language request that was not decoded;

or

identify an ambiguous decoding of a slot of the semantic representation.

12. A method for interpreting natural language requests, the method comprising:

generating, by a computing device, a semantic model as a function of a corpus of predefined requests, wherein the semantic model includes a plurality of mappings between a natural language request and a semantic representation of the natural language request, wherein the semantic representation identifies a user intent and zero or more slots associated with the user intent;

generating, by the computing device using the semantic model, a lattice of candidate alternatives indicative of a natural language request, wherein each candidate alternative represents a token of the natural language request, and wherein the lattice comprises a plurality of routes, wherein each route represents the natural language request;

assigning, by the computing device, a composite confidence weight to each candidate alternative as a function of the semantic model;

determining, by the computing device, an optimal route through the candidate alternative lattice from the plurality of routes based on the composite confidence weight; and

generating, by the computing device, a semantic representation of the natural language request as a function of the candidate alternatives of the optimal route;

wherein generating the semantic model comprises;

(i) identifying a contextual semantic feature in the corpus using a semi-supervised algorithm, wherein the contextual semantic feature comprises a sequence of lexical sets associated with a user intent and zero or more slots associated with the user intent;

(ii) determining a first probability of the contextual semantic feature given the user intent; and

(iii) determining a normalized probability of the user intent as a function of a rate of occurrence of the contextual semantic feature in the corpus; and

wherein identifying the contextual semantic feature using the semi-supervised algorithm comprises;

(i) tagging a first group of predefined sample queries in the corpus to identify user intents and slots;

(ii) extracting the contextual semantic feature from the sample queries;

(iii) analyzing, using the semantic model, a second group of predefined sample queries in the corpus;

(iv) extracting additional contextual semantic features in response to analyzing the second group of predefined sample queries; and

(v) updating the first probability and the normalized probability in response to analyzing the second group of predefined sample queries.

13. The method of claim 12, wherein generating the semantic model comprises:

generating an ontological index as a function of a predefined ontology associated with the user intent, wherein the ontology includes a plurality of objects describing a knowledge domain.

14. The method of claim 12, wherein generating the lattice comprises:

generating a candidate alternative corresponding to a user intent and associated slots of a mapping of the semantic model; and

generating a candidate alternative using a language model, as a function of phonetic similarity to the natural language request.

15. The method of claim 12, further comprising:

determining, by the computing device, whether the semantic representation of the natural language request includes sufficient information to perform a user intent of the semantic representation;

performing, by the computing device, the user intent in response to determining that the semantic representation includes sufficient information;

generating, by the computing device, a response as a function of the semantic representation;

generating, by the computing device, a natural language representation of the response using the semantic model; and

recording, by the computing device, a user dialog session including the natural language request and the natural language representation of the response into a corpus of recorded user dialog sessions.

16. The method of claim 12, further comprising:

updating, by the computing device, the semantic model in response to generating the semantic representation.

17. One or more non-transitory computer-readable storage media comprising a plurality of instructions that in response to being executed cause a computing device to:

generate a semantic model as a function of a corpus of predefined requests, wherein the semantic model includes a plurality of mappings between a natural language request and a semantic representation of the natural language request, wherein the semantic representation identifies a user intent and zero or more slots associated with the user intent;

generate, using the semantic model, a lattice of candidate alternatives indicative of a natural language request, wherein each candidate alternative represents a token of the natural language request, and wherein the lattice comprises a plurality of routes, wherein each route represents the natural language request;

assign a composite confidence weight to each candidate alternative as a function of the semantic model;

determine an optimal route through the candidate alternative lattice from the plurality of routes based on the composite confidence weight; and

generate a semantic representation of the natural language request as a function of the candidate alternatives of the optimal route;

wherein to generate the semantic model comprises to;

(i) identify a contextual semantic feature in the corpus using a semi-supervised algorithm, wherein the contextual semantic feature comprises a sequence of lexical sets associated with the user intent and zero or more slots associated with the user intent;

(ii) determine a first probability of the contextual semantic feature given the user intent; and

(iii) determine a normalized probability of the user intent as a function of a rate of occurrence of the contextual semantic feature in the corpus; and

wherein to identify the contextual semantic feature using the semi-supervised algorithm comprises to;

(i) tag a first group of predefined sample queries in the corpus to identify user intents and slots;

(ii) extract the contextual semantic feature from the sample queries;

(iii) analyze, using the semantic model, a second group of predefined sample queries in the corpus;

(iv) extract additional contextual semantic features in response to analyzing the second group of predefined sample queries; and

(v) update the first probability and the normalized probability in response to analyzing the second group of predefined sample queries.

18. The one or more non-transitory computer-readable storage media of claim 17, wherein to generate the semantic model comprises to:

generate an ontological index as a function of a predefined ontology associated with the user intent, wherein the ontology includes a plurality of objects describing a knowledge domain.

19. The one or more non-transitory computer-readable storage media of claim 17, wherein to generate the lattice comprises to:

generate a candidate alternative corresponding to the user intent and associated slots of a mapping of the semantic model; and

generate a candidate alternative using a language model, as a function of phonetic similarity to the natural language request.

20. The one or more non-transitory computer-readable storage media of claim 17, further comprising a plurality of instructions that in response to being executed cause the computing device to:

determine whether the semantic representation of the natural language request includes sufficient information to perform the user intent of the semantic representation;

perform the user intent in response to determining that the semantic representation includes sufficient information;

generate a response as a function of the semantic representation;

generate a natural language representation of the response using the semantic model; and

record a user dialog session including the natural language request and the natural language representation of the response into a corpus of recorded user dialog sessions.

21. The one or more non-transitory computer-readable storage media of claim 17, further comprising a plurality of instructions that in response to being executed cause the computing device to update the semantic model in response to generating the semantic representation.