Method and apparatus for operating a frequently asked questions (FAQ)-based system

• US 9,117,194 B2
• Filed: 12/06/2011
• Issued: 08/25/2015
• Est. Priority Date: 12/06/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Liu, Ding
• Primary Examiner(s): Hu, Jensen
• Application Number:
  US13/312,688
• Publication Number:
  US 20130144890A1
• Time in Patent Office: 1,358 Days
• Field of Search: 707/749
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Typical frequently-asked question (FAQ) systems perform keyword searches on queries to a database of answers to deliver an answer to a user. The present invention searches a user generated query in a classifier trained to search a database of questions in a set of FAQs. The questions in the database correspond to answers. Upon finding a relevant question, the system returns the corresponding answer to the user. Searching the query against the questions generates more accurate results for the user.

Claims

1. A method for operating a frequently asked questions (FAQ)-based system, the method comprising:

training a classifier primarily based on questions stored in the FAQ-based system, each question corresponding to at least one answer; and

applying the classifier to a query to return an answer, wherein applying the classifier is performed as a function of a context identified by a given sequence of queries, the classifier determining whether to update the context to reflect a most recent query in the given sequence of queries or reset the context based on an analysis of (i) the most recent query in the given sequence of queries in connection with the context and (ii) the most recent query in the given sequence of queries independent of the context.

2. The method of claim 1, further comprising determining relevance of words with respect to corresponding questions;

and

wherein training the classifier includes providing selected words, selected by a weight that represents the relevance of the words to the questions, as features to the classifier, the selected words being chosen from among the words having higher relevance than other words.

3. The method of claim 1, wherein the questions are preexisting questions, and wherein the method further comprises:

determining relevance of words within the answers with respect to corresponding questions; and

automatically generating supplemental questions based on words having higher relevance than other words, the supplemental questions corresponding to answers from which words of the supplemental questions were chosen; and

wherein training the classifier includes training the classifier based on the preexisting questions and the supplemental questions.

4. The method of claim 3, wherein generating the supplemental questions further includes:

ranking words within the corresponding answers as a function of relevance to corresponding questions; and

incorporating a given number of the ranked words into the preexisting questions to create the supplemental questions.

5. The method of claim 4, wherein the given number of ranked words is optionally adjustable or static.

6. The method of claim 1, wherein applying the classifier to return the answer includes returning a plurality of answers to a user.

7. The method of claim 6, further comprising adjusting rankings of the questions corresponding to answers, based on a given answer chosen by the user from among the plurality of answers returned, for use in returning answers to similar or the same queries in the future.

8. The method of claim 1, further comprising enabling receipt of the query from a human user or enabling receipt of the query from a machine user.

9. The method of claim 1, further comprising:

estimating a confidence level of the answer returned with respect to the query; and

enabling an override to allow a human to provide an alternative answer in an event the estimated confidence level of the returned answer is below a given threshold.

10. The method of claim 1, wherein the query is a formerly received query, and further including identifying a change in context in the most recently received query and resetting the given sequence of queries, from which the context is based, to begin a new context with the newest query.

11. The method of claim 10, further including applying weightings of decreasing value to words, used as factors for iteratively training the classifier, from within newest queries to within oldest queries in the given sequence of queries.

23. The method of claim 6, wherein training the classifier includes training the classifier in part as a function of a given answer chosen by the user.

24. The method of claim 1, wherein training the classifier further includes determining a highest ranking set of answers, creating features to provide to the classifier based on the queries and words of the highest ranking set of answers.

25. The method of claim 1, wherein the training is based on a combination of questions and answers.

26. The method of claim 25, wherein the training provides a higher weight to the questions than the answers.

12. A frequently asked questions (FAQ)-based system comprising:

a training module configured to train, using a processor, a classifier primarily based on questions stored in a memory of the FAQ-based system, each question corresponding to at least one answer; and

a query interface module configured to apply the classifier to a query to return an answer, wherein applying the classifier is performed as a function of context identified by a given sequence of queries, the classifier configured to determine whether to update the context to reflect a most recent query in the given sequence of queries or reset the context based on an analysis of (i) the most recent query in the given sequence of queries in connection with the context and (ii) the most recent query in the given sequence of queries independent of the context.

13. The system of claim 12, further comprising:

a relevance determination module configured to determine relevance of words with respect to corresponding questions;

wherein the training module is further configured to provide words selected by a weight that represents the relevance of the words to the questions as features to the classifier, the selected words being chosen from among the words having higher relevance than other words.

14. The system of claim 12, wherein the questions are preexisting questions;

and further comprising;

a relevance determination module configured to determine relevance of words within the answers with respect to corresponding questions;

a supplemental questions generation module configured to generate supplemental questions automatically based on words having higher relevance than other words, the supplemental questions corresponding to answers from which words of the supplemental questions were chosen; and

wherein the training module is configured to train the classifier based on the preexisting questions and the supplemental questions.

15. The system of claim 14, wherein the supplemental question generation module is further configured to rank words within the corresponding answers as a function of relevance to corresponding questions and incorporate a given number of the ranked words into the preexisting questions to create supplemental questions.

16. The system of claim 15, wherein the given number of ranked words is optionally adjustable or static.

17. The system of claim 12, wherein the query interface module is further configured to return a plurality of answers to a user.

18. The system of claim 17, wherein the training module is further configured to adjust rankings of questions corresponding to answers based on a given answer chosen by the user from among the plurality of answers returned for use in returning answers to similar or the same queries in the future.

19. The system of claim 12, wherein the query interface module is configured to enable receipt of the query from a human user or enable receipt of the query from a machine user.

20. The system of claim 12, wherein the query interface module is configured to estimate a confidence level of the answer returned with respect to the query, and further comprising:

an override module configured to allow a human to provide an alternative answer in the event the estimated confidence level of the returned answer is below a given threshold.

21. The system of claim 12, wherein the query is a formerly received query and further comprising:

a context switcher configured to identify a change in the context in the most recently received query and reset the given sequence of queries from which the context is based to begin a new context with the newest query.

22. The system of claim 21, wherein the context switcher further applies weightings of decreasing value to words used as factors for iteratively training the classifier from within newest queries to within oldest queries in the given sequence of queries.

27. The system of claim 17, wherein the training module is further configured to train the classifier in part as a function of a given answer chosen by the user.

28. The system of claim 12, wherein the query interface module is configured to determine a highest ranking set of answers and create features to provide to the classifier based on the queries and words of the highest ranking set of answers.

29. The system of claim 12, wherein the training module bases the training on a combination of questions and answers.

30. The system of claim 29, wherein the training module is configured to provide a higher weight to the questions than the answers.