Search and knowledge base question answering for a voice user interface

• US 10,331,402 B1
• Filed: 09/18/2017
• Issued: 06/25/2019
• Est. Priority Date: 05/30/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Spector, Daniel Lewis, O'Donoghue, Fergus, Brown, Jr., Chase Wesley, Snow, Shayne Leon, Horst, Brandon Gerald Li, Barton, William Folwell
• Primary Examiner(s): Guerra-Erazo, Edgar X
• Application Number:
  US15/707,496
• Time in Patent Office: 645 Days
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A voice-controlled question answering system that is capable of answering questions using both a knowledge base and a search engine. The knowledge base is used to answer questions when answers to those questions are contained in the knowledge base. If an answer using the knowledge base is unavailable, and if the question is suitable for answering using an unstructured search approach, the system may obtain an answer using a search engine. The search engine results may be processed to obtain an answer to the question suitable for output using a voice user interface.

Claims

1. A computer implemented method comprising:

receiving audio data representing a question input to a device in communication with a question answering system;

performing speech recognition on the audio data to obtain query text data;

determining feature data representing characteristics of the query text data;

processing the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search;

sending the query text data to a search component;

receiving, from the search component, a plurality of links corresponding to search results, wherein a first link of the plurality of links corresponds to a web address;

using the first link to obtain potential answer text from a web source;

selecting a sentence of the potential answer text;

performing natural language understanding (NLU) on the query text data to determine NLU results, wherein the performing NLU is performed at least partially in parallel with one or more of the processing, sending, receiving, using or selecting;

determining that the NLU results correspond to a request for an answer to the question;

generating output text including the sentence and an indicator of the web source;

performing text-to-speech processing on the output text to obtain output audio data; and

sending the output audio data to the device.

2. The computer implemented method of claim 1, further comprising:

determining that the query text data includes a first question word; and

determining that the first question word is in a defined group of approved question words,

wherein processing the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search further comprises;

processing the feature data with a first trained machine learning model to determine a question topic corresponding to the question text, and

determining that the question topic is in a defined group of approved question topics.

3. The computer implemented method of claim 2, wherein processing the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search further comprises:

processing the feature data with a second trained machine learning model to determine a score corresponding to how likely an unstructured web search is to obtain a correct answer to the question; and

determining that the score is above a threshold value.

4. The computer implemented method of claim 1, further comprising, prior to sending the output data:

using at least one second trained machine learning model, the sentence, and the query text data to determine that the sentence sufficiently corresponds to an answer to the user'"'"'s question.

5. A computer implemented method comprising:

determining input text data representing a question input to a device in communication with a question answering system;

determining feature data corresponding to characteristics of the input text data;

processing the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search;

sending the input text data to a search component;

receiving search results from the search component;

determining answer text data using the search results;

performing natural language understanding (NLU) on the input text data to determine NLU results;

determining that the NLU results correspond to a request for an answer to the question;

generating output text data based at least in part on the answer text data;

determining output audio data using the output text data; and

causing the output audio data to be output using a device.

6. The computer implemented method of claim 5, wherein the search results comprise a plurality of links and the method further comprises:

using a link of the plurality of links to obtain potential answer text data from a web source; and

selecting a sentence of the potential answer text data as the answer text data.

7. The computer implemented method of claim 5, further comprising:

determining that the input text data includes a first question word; and

determining that the first question word is in a group of approved question words.

8. The computer implemented method of claim 5, further comprising:

determining that the input text data does not include any words in a group of unapproved question words;

determining second input text data representing a second question;

determining that the second input text data includes at least one word in the group of unapproved question words; and

causing further output data to be output, the further output data indicating that a second question is not answerable.

9. The computer implemented method of claim 5, wherein processing the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search comprises:

processing the feature data with a first trained machine learning model to determine a question topic corresponding to the input text data; and

determining that the question topic is in a group of approved question topics.

10. The computer implemented method of claim 5, wherein processing the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search comprises:

processing the feature data with a first trained machine learning model to determine a score corresponding to how likely an unstructured web search is to obtain a correct answer to the question; and

determining that the score is above a threshold value.

11. The computer implemented method of claim 5, further comprising, prior to causing the output audio data to be output:

determining a knowledge base query from the input text data;

sending the knowledge base query to a knowledge base; and

determining that the question is not answerable using the knowledge base.

12. The computer implemented method of claim 5, further comprising:

determining indicator text corresponding to a source of the answer text data,

wherein determining the output audio data further uses the indicator text.

13. The computer implemented method of claim 5, further comprising:

using at least one second trained machine learning model, the answer text data, and the input text data to determine that the answer text data sufficiently corresponds to an answer to the question.

14. The computer implemented method of claim 5, wherein the performing NLU is performed at least partially in parallel with one or more of the processing, sending, receiving, or determining answer text data.

15. A computing system comprising:

at least one processor; and

memory including instructions operable to be executed by the at least one processor to configure the system to;

determine input text representing a question input to a device in communication with the system;

determine feature data corresponding to characteristics of the input text;

process the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search;

in response to determining the question is suitable for answering using the unstructured web search, send the input text to a search component;

receive search results from the search component;

determine answer text using the search results;

configure output text including the answer text;

determine output audio data using the output text; and

cause the output audio data to be output using a voice user interface.

16. The computing system of claim 15, wherein the search results comprise a plurality of links and the system is further configured to:

use a link of the plurality of links to obtain potential answer text from a web source; and

select a sentence of the potential answer text as the answer text.

17. The computing system of claim 15, the system further configured to:

determine that the input text includes a first question word; and

determine that the first question word is in a group of approved question words.

18. The computing system of claim 15, the system further configured to:

determine that the input text does not include includes any words in a group of unapproved question words.

19. The computing system of claim 15, wherein the system being configured to process the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search comprises the system being configured to:

process the feature data with a first trained machine learning model to determine a question topic corresponding to the input text; and

determine that the question topic is in a group of approved question topics.

20. The computing system of claim 15, wherein the system being configured to process the feature data using at least one trained machine learning model to determine that the question is suitable for answering using an unstructured web search comprises the system being configured to:

process the feature data with a first trained machine learning model to determine a score corresponding to how likely an unstructured web search is to obtain a correct answer to the question; and

determine that the score is above a threshold value.