Reranking QA answers using language modeling

• US 7,856,350 B2
• Filed: 08/11/2006
• Issued: 12/21/2010
• Est. Priority Date: 08/11/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chen, Yi, Zhou, Ming
• Primary Examiner(s): Sked; Matthew J
• Application Number:
  US11/502,859
• Publication Number:
  US 20080040114A1
• Time in Patent Office: 1,593 Days
• Field of Search: None
• US Class Current: 704/9
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

In a QA (Question/Answer) system, candidate answers in response to a question received are ranked by probabilities estimated by a language model. The language model is created based on an ordered centroid created from the question and information learned from an information source such as the Internet.

Claims

1. A computer implemented method for building a language model for use in a QA system comprising:

receiving a question, comprising a definitional question or a factoid, and determining a question focus of the question;

querying a source of information with the question focus and obtaining one or more relevant documents;

generating a centroid vector based on the question focus and said one or more relevant documents;

generating an ordered centroid based on the centroid vector; and

utilizing a computer processor that is a component of a computing device to build the language model based on the ordered centroid.

2. The computer implemented method of claim 1 wherein the question is a definitional question.

3. The computer implemented method of claim 1 wherein the question is a factoid.

4. The computer implemented method of claim 1 wherein the language model is an n-gram model.

5. The computer implemented method of claim 4 wherein the n-gram model is a bi-gram model.

6. The computer implemented method of claim 4 wherein the n-gram model is a bi-term model.

7. The computer implemented method of claim 1 wherein building the language model comprises:

building the language model using co-occurring terms with the question focus.

8. The computer implemented method of claim 1 wherein building the language model comprises:

obtaining relevant sentences and/or phrases having the question focus and one or more co-occurring terms.

9. The computer implemented method of claim 1 wherein obtaining relevant sentences comprises querying a second source of information.

10. The computer implemented method of claim 9 wherein the second source of information comprises the Internet.

11. A computer implemented method for building a language model for use in a QA system receiving definitional questions, the method comprising:

receiving a definitional question and determining a question focus of the definitional question;

obtaining information relevant to the question focus;

generating a centroid vector based on the question focus and the information relevant to the question focus;

generating an ordered centroid based on centroid vector; and

building the language model based on the ordered centroid.

12. The computer implemented method of claim 11 wherein obtaining information includes searching a source of information using a search engine and a query comprising the question focus and clue words based on a type of the definitional question.

13. The computer implemented method of claim 12 wherein obtaining information includes:

receiving results based on the query comprising the question focus and clue words based on the type of the definitional question;

processing the results to obtain expansion terms; and

querying the source of information with the question focus and selected expansion terms based on the results; and

wherein generating the centroid vector comprises using the results from querying the source of information with the question focus and selected expansion terms based on the results.

14. The computer implemented method of claim 13 wherein generating the centroid vector includes generating the centroid vector based on co-occurring terms proximate the question focus in phrases and/or sentences in the results.

15. The computer implemented method of claim 11 and using the language model to rerank candidate answers in response to a definitional question in a QA system.

16. A computer readable medium having instructions, which when executed by a computer, implement a QA system that builds a language model, the instructions comprising:

receiving a definitional question and determining a question focus of the definitional question;

querying a source of information with the question focus and obtaining one or more relevant documents;

generating a centrold vector based on the question focus and said one or more relevant documents;

generating an ordered centroid based on the centroid vector; and

utilizing a computer processor that is a component of a computing, device to build the language model based on the ordered centroid.

17. The computer readable medium of claim 16 wherein the language model is n-gram model.