System and methods for a virtual assistant to manage and use context in a natural language dialog

• US 10,418,032 B1
• Filed: 05/24/2016
• Issued: 09/17/2019
• Est. Priority Date: 04/10/2015
• Status: Active Grant

• Current Assignee: Soundhound AI IP Holding LLC (SoundHound AI, Inc. (f/k/a Archimedes Tech SPAC Partners Co.)), Soundhound AI IP LLC (SoundHound AI, Inc. (f/k/a Archimedes Tech SPAC Partners Co.))
• Original Assignee: SoundHound, Inc. (SoundHound AI, Inc. (f/k/a Archimedes Tech SPAC Partners Co.))
• Inventors: Mohajer, Keyvan, Wilson, Christopher, Mont-Reynaud, Bernard, Collecchia, Regina
• Primary Examiner(s): Albertalli, Brian L
• Application Number:
  US15/163,485
• Time in Patent Office: 1,211 Days
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A dialog with a conversational virtual assistant includes a sequence of user queries and systems responses. Queries are received and interpreted by a natural language understanding system. Dialog context information gathered from user queries and system responses is stored in a layered context data structure. Incomplete queries, which do not have sufficient information to result in an actionable interpretation, become actionable with use of context data. The system recognizes the need to access context data, and retrieves from context layers information required to transform the query into an executable one. The system may then act on the query and provide an appropriate response to the user. Context data buffers forget information, perhaps selectively, with the passage of time, and after a sufficient number and type of intervening queries.

Claims

1. A method of disambiguating interpretations, the method comprising:

receiving from a client a natural language query and an associated dialog history, the natural language query having an incomplete interpretation, the incomplete interpretation having at least one slot name whose value is ambiguous, the associated dialog history including one or more entries for history items, each history item including a previous interpretation and being associated with a named slot from the set comprising WHO, WHAT, WHEN, WHERE and HOW MANY, and each history item entry being associated with a slot value, saliency weight and sequence number;

selecting, from the associated dialog history, a history item including a previous interpretation, the history item being selected, from the associated dialog history, based on the slot name of the history item, the number sequence of the history item and the saliency weight of the history item, such that the selected history item matches the natural language query having the incomplete interpretation for purposes of completing the at least one slot name whose value is ambiguous;

merging the previous interpretation of the selected history item with the incomplete interpretation to form a merged interpretation;

creating an updated dialog history by adding, to the associated dialog history, a new dialog history layer comprising the merged interpretation and at least one additional history item having a slot name, a slot value, a saliency weight, and a new sequence number; and

sending the updated dialog history to the client.

2. The method of claim 1 wherein at least one of the history items of the associated dialog history stores a variable-length array of values.

3. The method of claim 1 wherein the set is open-ended, the method further comprises creating a new history item within the new dialog history layer, and the new history item has a new slot name.

4. The method of claim 1 wherein:

the natural language query received from the client is a stream of data and the associated dialog history is received at a beginning of the natural language query; and

the updated dialog history sent to the client is part of structured output.

5. The method of claim 1, wherein the matching history item is selected by (i) comparing the slot name of each history item with the at least one slot name whose value is ambiguous, (ii) considering the saliency weight for each history item, where the saliency weight indicates an importance of each respective history item in the associated dialog history and (iii) considering the sequence number of each history item, where the sequence number indicates a time sequence in which each respective history item has been placed in the associated dialog history.

6. The method of claim 5, wherein the matching history item is selected by (i) identifying one or more history items, from the associated dialog history, having a slot name that matches the at least one slot name whose value is ambiguous and (ii) selecting, as the selected matching history item, one of the identified one or more history items based on the saliency weights and the sequence numbers of the identified one or more history items.

7. A non-transitory computer-readable recording medium having computer program instructions recorded thereon, the computer program instructions, when executed on a computer processor, causing the computer processor to perform a method of disambiguating interpretations, the method comprising:

receiving from a client a natural language query and an associated dialog history, the natural language query having an incomplete interpretation, the incomplete interpretation having at least one slot name whose value is ambiguous, the associated dialog history including one or more entries for history items, each history item including a previous interpretation and being associated with a named slot from the set comprising WHO, WHAT, WHEN, WHERE and HOW MANY, and each history item entry being associated with a slot value, saliency weight and sequence number;

selecting, from the associated dialog history, a history item including a previous interpretation, the history item being selected, from the associated dialog history, based on the slot name of the history item, the number sequence of the history item and the saliency weight of the history item, such that the selected history item matches the natural language query having the incomplete interpretation for purposes of completing the at least one slot name whose value is ambiguous;

merging the previous interpretation of the selected history item with the incomplete interpretation to form a merged interpretation;

creating an updated dialog history by adding, to the associated dialog history, a new dialog history layer comprising the merged interpretation and at least one additional history item having a slot name, a slot value, a saliency weight, and a new sequence number; and

sending the updated dialog history to the client.

8. The non-transitory computer-readable recording medium of claim 7, wherein the matching history item is selected by (i) comparing the slot name of each history item with the at least one slot name whose value is ambiguous, (ii) considering the saliency weight for each history item, where the saliency weight indicates an importance of each respective history item in the associated dialog history and (iii) considering the sequence number of each history item, where the sequence number indicates a time sequence in which each respective history item has been placed in the associated dialog history.

9. The non-transitory computer-readable recording medium of claim 8, wherein the matching history item is selected by (i) identifying one or more history items, from the associated dialog history, having a slot name that matches the at least one slot name whose value is ambiguous and (ii) selecting, as the selected matching history item, one of the identified one or more history items based on the saliency weights and the sequence numbers of the identified one or more history items.

10. A system including one or more processors coupled to memory, the memory loaded with computer program instructions for disambiguating interpretations, the computer program instructions, when executed on the processors, implement actions comprising

receiving from a client a natural language query and an associated dialog history, the natural language query having an incomplete interpretation, the incomplete interpretation having at least one slot name whose value is ambiguous, the associated dialog history including one or more entries for history items, each history item including a previous interpretation and being associated with a named slot from the set comprising WHO, WHAT, WHEN, WHERE and HOW MANY, and each history item entry being associated with a slot value, saliency weight and sequence number;

selecting, from the associated dialog history, a history item including a previous interpretation, the history item being selected, from the associated dialog history, based on the slot name of the history item, the number sequence of the history item and the saliency weight of the history item, such that the selected history item matches the natural language query having the incomplete interpretation for purposes of completing the at least one slot name whose value is ambiguous;

merging the previous interpretation of the selected history item with the incomplete interpretation to form a merged interpretation;

creating an updated dialog history by adding, to the associated dialog history, a new dialog history layer comprising the merged interpretation and at least one additional history item having a slot name, a slot value, a saliency weight, and a new sequence number; and

sending the updated dialog history to the client.

11. The system of claim 10, wherein the matching history item is selected by (i) comparing the slot name of each history item with the at least one slot name whose value is ambiguous, (ii) considering the saliency weight for each history item, where the saliency weight indicates an importance of each respective history item in the associated dialog history and (iii) considering the sequence number of each history item, where the sequence number indicates a time sequence in which each respective history item has been placed in the associated dialog history.

12. The system of claim 11, wherein the matching history item is selected by (i) identifying one or more history items, from the associated dialog history, having a slot name that matches the at least one slot name whose value is ambiguous and (ii) selecting, as the selected matching history item, one of the identified one or more history items based on the saliency weights and the sequence numbers of the identified one or more history items.