Search and knowledge base question answering for a voice user interface

• US 10,642,577 B2
• Filed: 06/18/2019
• Issued: 05/05/2020
• Est. Priority Date: 05/30/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Spector, Daniel Lewis, O'Donoghue, Fergus, Brown, Jr., Chase Wesley, Snow, Shayne Leon, Horst, Brandon Gerald Li, Barton, William Folwell
• Primary Examiner(s): Guerra-Erazo, Edgar X
• Application Number:
  US16/444,741
• Publication Number:
  US 20190369957A1
• Time in Patent Office: 322 Days
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A voice-controlled question answering system that is capable of answering questions using both a knowledge base and a search engine. The knowledge base is used to answer questions when answers to those questions are contained in the knowledge base. If an answer using the knowledge base is unavailable, and if the question is suitable for answering using an unstructured search approach, the system may obtain an answer using a search engine. The search engine results may be processed to obtain an answer to the question suitable for output using a voice user interface.

Claims

1. A computer-implemented method comprising:

receiving input audio data representing a question input to a question answering system;

performing speech recognition on the input audio data to obtain first text data;

determining feature data related to the question;

sending at least a portion of the first text data to a search component;

determining second text data corresponding to a plurality of search results;

processing the feature data and the second text data using a first machine learning model to determine third text data represents an appropriate answer to the question;

performing text-to-speech processing using at least a portion of the third text data to obtain output audio data; and

sending the output audio data to a device.

2. The computer-implemented method of claim 1, further comprising:

performing natural language understanding (NLU) on the first text data to determine NLU result data,

wherein the feature data comprises the NLU result data.

3. The computer-implemented method of claim 1, further comprising:

selecting a portion of the second text data to include in the third text data.

4. The computer-implemented method of claim 1, further comprising:

receiving a plurality of links corresponding to the plurality of search results,

wherein;

determining the second text data comprises;

using a first link of the plurality of links to obtain first potential answer text data from a first web source, and

using a second link of the plurality of links to obtain second potential answer text data from a second web source; and

the second text data includes at least the first potential answer text data and the second potential answer text data; and

the method further comprises selecting a portion of the first potential answer text data as the third text data.

5. The computer-implemented method of claim 4, wherein:

processing the feature data and the second text data using a first machine learning model comprises;

determining a first score corresponding to the first potential answer text data,

determining a second score corresponding to the second potential answer text data; and

the method further comprises determining the first score corresponds to an appropriate answer to the question.

6. The computer-implemented method of claim 4, further comprising:

determining indicator text corresponding to the first web source,

wherein performing the text-to-speech processing further uses the indicator text to obtain the output audio data that includes a representation of the first web source.

7. The computer-implemented method of claim 4, wherein using the first link of the plurality of links to obtain the first potential answer text data comprises:

using the first link to obtain fourth text data from the first web source, the fourth text data including a first portion and a second portion;

determining that the second portion does not correspond to an answer to the question; and

selecting the first portion as the first potential answer text data.

8. The computer-implemented method of claim 1, wherein determining the feature data comprises including at least a portion of the first text data in the feature data.

9. The computer-implemented method of claim 1, further comprising:

determining syntactic information corresponding to the question; and

selecting fourth text data based at least in part on the syntactic information,

wherein performing the text-to-speech processing further uses the fourth text data.

10. The computer-implemented method of claim 1, further comprising:

processing the feature data using at least one second trained machine learning model to determine that the question is suitable for answering using an unstructured web search.

11. A system comprising:

at least one processor; and

memory including instructions operable to be executed by the at least one processor to configure the system to;

receive input audio data representing a question input to a question answering system;

perform speech recognition on the input audio data to obtain first text data;

determine feature data related to the question;

send at least a portion of the first text data to a search component;

determine second text data corresponding to a plurality of search results;

process the feature data and the second text data using a first machine learning model to determine third text data represents an appropriate answer to the question;

perform text-to-speech processing using at least a portion of the third text data to obtain output audio data; and

send the output audio data to a device.

12. The system of claim 11, wherein the memory further includes instructions that are operable to be executed by the at least one processor to further configure the system to:

perform natural language understanding (NLU) on the first text data to determine NLU result data,

wherein the feature data comprises the NLU result data.

13. The system of claim 11, wherein the memory further includes instructions that are operable to be executed by the at least one processor to further configure the system to:

select a portion of the second text data as the third text data.

14. The system of claim 11, wherein the memory further includes instructions that are operable to be executed by the at least one processor to further configure the system to:

receive a plurality of links corresponding to the plurality of search results,

wherein;

the instructions that are operable to be executed by the at least one processor to configure the system to determine the second text data comprise instructions that are operable to be executed by the at least one process to configure the system to;

use a first link of the plurality of links to obtain first potential answer text data from a first web source, and

use a second link of the plurality of links to obtain second potential answer text data from a second web source; and

the second text data includes at least the first potential answer text data and the second potential answer text data; and

the memory further includes instructions that are operable to be executed by the at least one processor to further configure the system to select a portion of the first potential answer text data as the third text data.

15. The system of claim 14, wherein the instructions that are operable to be executed by the at least one processor to configure the system to process the feature data and the second text data using a first machine learning model comprise instructions that are operable to be executed by the at least one processor to configure the system to:

determine a first score corresponding to the first potential answer text data;

determine a second score corresponding to the second potential answer text data; and

the memory further includes instructions that are operable to be executed by the at least one processor to further configure the system to determine the first score corresponds to an appropriate answer to the question.

16. The system of claim 14, wherein the memory further includes instructions that are operable to be executed by the at least one processor to further configure the system to:

determine indicator text corresponding to the first web source,

wherein the instructions that are operable to be executed by the at least one processor to configure the system to perform the text-to-speech processing further uses the indicator text to obtain output audio data that includes a representation of the first web source.

17. The system of claim 14, wherein the instructions that are operable to be executed by the at least one processor to configure the system to use the first link of the plurality of links to obtain the first potential answer text data comprise instructions that are operable to be executed by the at least one processor to configure the system to:

use the first link to obtain fourth text data from the first web source, the fourth text data including a first portion and a second portion;

determine that the second portion does not correspond to an answer to the question; and

select the first portion as the first potential answer text data.

18. The system of claim 14, wherein the instructions that are operable to be executed by the at least one processor to configure the system to determine the feature data comprise instructions that are operable to be executed by the at least one processor to configure the system to include at least a portion of the first text data in the feature data.

19. The system of claim 11, wherein the memory further includes instructions that are operable to be executed by the at least one processor to further configure the system to:

determine syntactic information corresponding to the question;

select fourth text data based at least in part on the syntactic information,

wherein the instructions that are operable to be executed by the at least one processor to configure the system to perform the text-to-speech processing further uses the fourth text data.

20. The system of claim 11, wherein the memory further includes instructions that are operable to be executed by the at least one processor to further configure the system to:

process the feature data using at least one second trained machine learning model to determine that the question is suitable for answering using an unstructured web search.