Knowledge extraction from online discussion forums

• US 7,814,048 B2
• Filed: 08/14/2006
• Issued: 10/12/2010
• Est. Priority Date: 08/14/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Huang, Jizhou, Zhou, Ming
• Primary Examiner(s): Trujillo; James
• Assistant Examiner(s): SHECHTMAN, CHERYL MARIA
• Application Number:
  US11/464,373
• Publication Number:
  US 20080046394A1
• Time in Patent Office: 1,520 Days
• Field of Search: 707/104.1, 707/5, 707/100, 707/602, 707/692, 707/723, 707/754, 718/100, 718/103, 715/733, 702/1, 706/12, 706/59, 709/204, 709/206, 705/1, 705/10, 705/500
• US Class Current: 707/602
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]

Abstract

Concepts presented herein relate to extracting knowledge for a chatbot knowledge base from online discussion forms. Within a thread of an online discussion form, replies are selected based on structural features and content features therein. The replies can be ranked and used in a chatbot knowledge base.

Claims

1. A method, comprising:

accessing a thread from a discussion forum having a plurality of threads, the accessed thread having a root message with a thread title and a plurality of replies associated with the root message;

selecting replies from the plurality of replies in the accessed thread by analyzing structural features and content features of each reply, wherein the structural features provide context of a given reply as related to other of the plurality of replies to the root message and the content features include words related to the root message;

applying a filter to remove one or more replies from the previously selected replies by comparing a keyword list having a plurality of words, wherein the keyword list includes words indicative of personal identifying information, to content features in each of the selected replies and removing those replies that have at least one of the words indicative of personal identifying information in its content features;

ranking the replies previously selected from the plurality of replies in the accessed thread that remain after applying the filter using a ranking model based on ranking features of the replies;

generating a list of replies from the selected replies based on results of the ranking; and

storing the list of replies in a data store to create a knowledge base for an automated conversational agent.

2. The method of claim 1 wherein the ranking features are based on determining how many replies provide a quotation from the reply to be ranked within its associated thread.

3. The method of claim 1 wherein the ranking features are based on an author of one of the selected replies.

4. The method of claim 3 wherein the ranking features are based on at least one of a number of threads started by the author in a discussion forum, a number of replies an author posts, an average length of replies posted by the author and a number of times an author is quoted in the thread.

5. The method of claim 1 and further comprising:

forming a chatbot knowledge base based on the selected replies.

6. The method of claim 1 wherein the structural features are indicative of a relationship between each reply and at least one of a root message in the accessed thread and another reply in the accessed thread.

7. The method of claim 1 wherein the content features are indicative of terms in a reply.

8. A method, comprising:

accessing a thread of a discussion forum having a root message and a plurality of replies to the root message;

identifying structural features for each reply of the plurality of replies, the structural features being indicative of a contextual relationship in the thread between a given reply and at least one of the root message and another reply of the plurality of replies;

identifying content features for each reply of the plurality of replies, the content features being indicative of terms used in the reply;

selecting replies from the plurality of replies based on the structural features and the content features for each reply;

applying a filter to remove one or more entire replies from the previously selected replies by comparing a keyword list having a plurality of words, including words that are indicative of personal identifying information to content features in each of the selected replies and removing those replies that have at least one of the words indicative of personal identifying information in its content features; and

ranking the selected replies that remain after applying the filter based on the structural features, content features, and labels applied to the replies, wherein the labels indicate a quality of the replies.

9. The method of claim 8 wherein the structural features relate to whether the reply includes text from the root message.

10. The method of claim 8 wherein the structural features relate to whether the reply includes text from another reply in the thread.

11. The method of claim 8 wherein the content features relate to a number of words in the reply.

12. The method of claim 8 and further comprising:

ranking the selected replies as a function of ranking features.

13. The method of claim 12 wherein the ranking features are based on a number of quotations a reply receives in a thread and an author of the reply.

14. A system having a computer readable storage medium storing a plurality of executable modules for processing threads in an online discussion forum, each thread having a title and a plurality of replies, the executable modules comprising:

an identification module that accesses the threads and selects some of the replies located within the threads as a function of structural features, the structural features being indicative of a relationship between replies in the context of the thread and content features contained within the replies;

a filter that removes one or more replies from the previously selected replies by comparing a keyword list having a plurality of words, including words that are indicative of personal information, to content features in each of the selected replies and removing those replies that have at least one of the words that are indicative of personal information in its content features;

a ranking module for ranking the previously selected replies based on ranking features contained therein and

a computer processor being a functional component of the system and activated by the identification module, the filter and the ranking module, to facilitate selecting, removing and ranking.

15. The system of claim 14 wherein the structural features are indicative of a relationship between each reply and at least one of a root message of the thread and another reply in the thread.

16. The system of claim 14 wherein the content features are indicative of terms in a reply.

17. The system of claim 14 wherein the ranking features are indicative of at least one of a number of quotations a reply receives within the thread and an author of the reply.