Categorization based text processing

• US 6,618,715 B1
• Filed: 06/08/2000
• Issued: 09/09/2003
• Est. Priority Date: 06/08/2000
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Johnson, David E., Damerau, Frederick J.
• Primary Examiner(s): Starks, Jr., Wilbert L.
• Assistant Examiner(s): Hirl, Joseph P.
• Application Number:
  US09/589,398
• Time in Patent Office: 1,188 Days
• Field of Search: 706/47, 706/45, 706/48, 707/500
• US Class Current: 706/47
• CPC Class Codes: G06F 16/35 Clustering; Classification

Abstract

A rules based configurable system efficiently and effectively determines for a given electronically represented text document which linguistic analysis and extraction processes and which application specific processes should be invoked to provide more accurate answers to a user'"'"'s query. In a rules based classifier, where each category or topic is represented by a set of rules, in an application such as routing, the categorization effecting the routing can be effectively combined with processes extracting other information. This may be in the form of a prompt for the user to input additional information.

Claims

1. A configurable system for determining for a given electronically represented text document which linguistic analysis and extraction processes and which application specific processes should be invoked to provide formatted answers to a user'"'"'s query comprising:

a categorizer receiving an input text document and classifying the text document into zero, one or more categories;

a processor selection system receiving a list of categories from the categorizer and determining which of a plurality of extractor processes to invoke on the input document based on categorization of the input text document into zero, one or more categories; and

an extractor system implementing the plurality of extractor processes invoked by the processor selection system for extracting data from the input text document and formatting the extracted data.

2. The configurable system recited in claim 1, wherein the categorizer is a rules based classifier, where each category or topic is represented by a set of rules.

3. The configurable system recited in claim 2, wherein the rules base classifier is a decision-tree-based rule induction system.

4. The configurable system recited in claim 1, wherein the document categorizations are associated with confidence levels.

5. The configurable system recited in claim 1, wherein the system is used in a routing application, further comprising a router receiving data from the extractor system and from the categorizer and routing the data to a message handler based on the output of the categorizer.

6. A computer implemented method of extracting formatted information from unformatted text files to provide formatted answers to a user'"'"'s query comprising the steps of:

receiving an input text document;

classifying the input text document into zero, one or more categories;

determining which of a plurality of extractor processes to invoke on the input document based on categorization of the input text document;

invoking one or more of said plurality of extractor processes; and

extracting data from the input text document and formatting the extracted data.

7. The method of extracting formatted information from unformatted text files to provide formatted answers to a user'"'"'s query recited in claim 6, wherein the step of classifying is based on a rules based classifier process, where each category or topic is represented by a set of rules.

8. The method as recited in claim 7, wherein each category or topic represented by a set of rules is associated with a confidence level.

9. The method of extracting formatted information from unformatted text files to provide formatted answers to a user'"'"'s query recited in claim 6, further comprising the steps of:

receiving formatted extracted data; and

routing the formatted extracted date to a message handler based on categorization of the input text document.

10. A method as recited in claim 6, wherein the step of determining which of a plurality of extractor processes to invoke on the input document based on categorization of the input text document, further comprises the steps of:

identifying a confidence level for each classified category of the input text document; and

selecting a subset of extractor processes associated with a 100% confidence level for each classified category of the input text document, where the plurality of extractor processes to invoke comprises the selected subsets of extractor processes.