Method and apparatus for compiling and querying a QA database

• US 8,412,514 B1
• Filed: 10/27/2005
• Issued: 04/02/2013
• Est. Priority Date: 10/27/2005
• Status: Active Grant

• Current Assignee: Shopify Incorporated
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Feng, Junlan, Gilbert, Mazin, Tur, Gokhan, Hakkani-Tr, Dilek
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US11/260,412
• Time in Patent Office: 2,714 Days
• Field of Search: None
• US Class Current: 704/9
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 40/194 Calculation of difference between files
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F

Abstract

Disclosed is a method and apparatus for responding to an inquiry from a client via a network. The method and apparatus receive the inquiry from a client via a network. Based on the inquiry, question-answer pairs retrieved from the network are analyzed to determine a response to the inquiry. The QA pairs are not predefined. As a result, the QA pairs have to be analyzed in order to determine whether they are responsive to a particular inquiry. Questions of the QA pairs may be repetitive and similar to one another even for very different subjects, and without additional contextual and meta-level information, are not useful in determining whether their corresponding answer responds to an inquiry.

Claims

1. A method for responding to an inquiry comprising:

retrieving a first web page and a second web page, the first web page associated with a first layout and the second web page associated with a second layout;

comparing the first layout and the second layout;

generating stored question-answer pairs based on the comparing the first layout and the second layout;

receiving the inquiry via a network;

based on the inquiry, analyzing the stored question-answer pairs;

determining meta-level features of the stored question-answer pairs to define relationships among question-answer pairs, the meta-level features of a particular stored question-answer pair based at least in part on the meta-level features of a previous related stored question-answer pair and a first related question-answer pair;

clarifying the inquiry based on follow-up procedures including asking follow-up questions;

determining a plurality of responses to the inquiry based on the analysis of the meta-level features, the relationships among the stored question-answer pairs, and the follow-up procedures;

providing the plurality of responses;

receiving a selection of a particular response from among the plurality of responses; and

storing a measure of the eligibility of the particular response to be a response to the inquiry based on the selection.

2. The method of claim 1 wherein the stored question-answer pairs are unstructured question-answer pairs.

3. The method of claim 1 further comprising retrieving the stored question-answer pairs from servers.

4. The method of claim 3 wherein the retrieving the first web page further comprises analyzing structure and content of the first web page to determine whether the first web page has a question-answer pair on it.

5. The method of claim 3 further comprising detecting frequently asked question templates on a web page.

6. The method of claim 3 further comprising detecting semantic blocks encoded with similar layout properties to locate question-answer pairs.

7. The method of claim 1, wherein the analyzing stored question-answer pairs further comprises linguistic text pre-processing to associate a linguistic tag with each of the stored question-answer pairs.

8. The method of claim 7, wherein-determining a response to the inquiry comprises analyzing the inquiry, measuring proximity between the inquiry and the stored question-answer pairs, and measuring eligibility of the stored question-answer pairs to be responses for the inquiry.

9. The method of claim 8 further comprising measuring the proximity and measuring the eligibility by considering the meta-level features and the associated linguistic tags of the stored question-answer pairs.

10. The method of claim 1, wherein the meta-level features of the particular stored question-answer pair further comprise a location of the particular stored question-answer pair, a resolution of a pronominal anaphora using preceding question-answer pairs, and relevant named entities.

11. The method of claim 1 wherein the inquiry comprises at least one of natural language questions, keyword search patterns, hand written text, spoken utterances, and spoken phrases.

12. The method of claim 1 wherein the follow-up procedures further comprise at least one of displaying multiple responses to the inquiry, enabling a rating of the multiple responses, and arriving at the response based on answers from a client to multiple responses.

13. A system for responding to an inquiry comprising:

a web miner configured to retrieve a first web page and a second web page, the first web page associated with a first layout and the second web page associated with a second layout and to compare the first layout and the second layout and to generate stored question-answer pairs based on the comparing the first layout and the second layout;

an interface configured to

receive the inquiry via a network;

provide a plurality of responses

receive a selection of a particular response from among the plurality of responses;

a database comprising stored question-answer pairs;

a feature extractor module for extracting features of the stored question-answer pairs from the database and labeling each question-answer pair with meta-level features to define relationships among stored question-answer pairs, the meta-level features of a particular stored question-answer pair based on the meta-level features of a previous related stored question-answer pair and a first related stored question-answer pair; and

a processor configured to;

clarify the inquiry based on follow-up procedures including asking follow-up questions and analyzing the inquiry to determine the plurality of responses to the inquiry based on the stored question answer pairs, the relationships among the stored question answer pairs, and the follow-up procedures; and

store a measure of the eligibility of the particular response to be a response to the inquiry based on the selection.

14. The system of claim 13 wherein the stored question-answer pairs further comprise unstructured question-answer pairs.

15. The system of claim 14 wherein the analyzing of the inquiry further comprises using at least one of voice recognition and handwriting recognition.

16. The system of claim 13, wherein the meta-level feature further comprises a location of the particular stored question-answer pair, resolution of a pronominal anaphora using preceding question-answer pairs, and relevant named entities.

17. The system of claim 16 wherein the relevant named entities comprise a company name, name of a person, a location, and time associated with the particular stored question-answer pair.

18. The system of claim 13 wherein the follow-up procedures further comprise at least one of displaying multiple responses to the inquiry, enabling a rating of the multiple responses, and arriving at the response based on answers from a client to multiple responses.

19. The system of claim 13 wherein the web miner analyzes structure and content of a web page to determine whether the web page has a question-answer pair on it.

20. The system of claim 13 wherein the web miner detects question-answer pair templates on a web page.

21. The system of claim 13 wherein the web miner detects semantic blocks encoded with similar layout properties to locate unstructured question-answer pairs.

22. The system of claim 13 wherein the web miner checks question-answer pair layouts on a web page of a site with other question-answer pair layouts on other web pages of the site.

23. A system for responding to an inquiry comprising:

means for retrieving a first web page and a second web page, the first web page associated with a first layout and the second web page associated with a second layout;

means for comparing the first layout and the second layout;

means for generating stored question-answer pairs based on the comparing the first layout and the second layout;

means for receiving the inquiry via a network;

means for analyzing stored question-answer pairs based on the inquiry;

means for determining meta-level features of the stored question-answer pairs to define relationships among the stored question-answer pairs, the meta-level features of a particular stored question-answer pair based at least in part on the meta-level features of a previous related stored question-answer pair and a first related stored question-answer pair;

means for clarifying the inquiry based on follow-up procedures including asking follow-up questions;

means for determining a plurality of responses to the inquiry based on the analysis of the stored question-answer pairs, the relationships among the stored question-answer pairs, and the follow-up procedures;

means for providing the plurality of responses;

means for receiving a selection of a particular response from among the plurality of responses; and

means for storing a measure of the eligibility of the particular response to be a response to the inquiry based on the selection.

24. The system of claim 23 wherein the stored question-answer pairs further comprise stored unstructured question-answer pairs.

25. The system of claim 24 wherein the stored unstructured question-answer pairs are retrieved from servers.

26. The system of claim 23, wherein the meta-level features further comprise a location of the stored question-answer pair, a resolution of a pronominal anaphora using preceding question-answer pairs, and relevant named entities.

27. The system of claim 23 wherein the follow-up procedures further comprise at least one of displaying multiple responses to the inquiry, enabling a rating of the multiple responses, and arriving at the response based on answers from a client to multiple responses.

28. The system of claim 23 further comprising means for analyzing structure and content of the first web page to determine whether the web page has a question-answer pair on it.

29. The system of claim 23 further comprising means for detecting question-answer pair templates on a web page.

30. The system of claim 23 further comprising means for detecting semantic blocks encoded with similar layout properties to locate unstructured question-answer pairs.

31. The system of claim 23 further comprising means for checking question-answer pair layouts on a web page of a site with other question-answer pair layouts on other web pages of the site.