Methodology for creating and maintaining a scheme for categorizing electronic communications

• US 6,925,454 B2
• Filed: 12/12/2000
• Issued: 08/02/2005
• Est. Priority Date: 12/12/2000
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Lam, Kathryn K., Oles, Frank J.
• Primary Examiner(s): Jeanty, Romain
• Application Number:
  US09/733,946
• Publication Number:
  US 20020107712A1
• Time in Patent Office: 1,694 Days
• Field of Search: 705/8, 706/45, 709/206
• US Class Current: 706/45
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06Q 10/0633 Workflow analysis
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Supervised learning is used to develop a computer program that can automatically route or respond to electronic communications assuming the existence of an appropriate formal scheme for categorizing incoming electronic communications. A method is described by which such a categorization scheme for electronic communications can be constructed. The method is based on an analysis of factors having an impact on the categorization scheme from both the business domain and the technology domain. The problem solved by this method is a new one that is only now emerging as automated methods of routing communications based on supervised learning are becoming feasible. Among other uses, this method may be practically employed as a disciplined way of carrying out consulting engagements that call for setting up and maintaining categorization schemes for electronic communications.

Claims

1. A computer implemented method for categorizing incoming electronic communications using a supervised machine learning component, and for factoring an organization'"'"'s business domain into the technology domain to enable an acceptable automated response and routing scheme, said method comprising the steps of:

(a) analyzing the business domain;

(b) determining an approach to machine learning using a program or an algorithm for inducing a categorizer using supervised learning, the categorizer being generated from training data comprising a set of examples of the type of electronic communications to be classified;

(c) collecting existing data of representative examples of electronic communications and inventories of personnel skills, business processes, workflows, and business missions;

(d) Analyzing the collected data for determining one or more attributes of said electronic communications selected from the group consisting of complexity, vagueness, and uniqueness to be expected in the type of communications to be categorized, as well as the relative numbers of electronic communications having a particular attribute of said one or more attributes, and for determining a technical structure of the communications relevant to categorization, and factoring the inventories of personnel skills, business processes, workflows, and business missions collected to determine what must be done with each electronic communication, and by whom;

(e) defining a categorization scheme;

(f) labeling examples of electronic communications with categories from the categorization scheme for use both as training data to be used in the supervised learning step and as test data;

(g) converting, using a computer, the labeled examples into a form suitable for subsequent processing, both for purposes of machine learning and technical validation;

(h) performing using said computer, machine based supervised learning technology to induce said categorizer for the categorization scheme; and

(i) validating the categorization scheme with respect to technical performance and business requirements.

2. A method as recited in claim 1, further comprising the step of implementing the categorization scheme by putting the categorization system into production.

3. A method as recited in claim 1, further comprising the steps of:

reviewing the categorization scheme to consider its adequacy in light of recent distribution of communications; and

modifying the categorization scheme, as required, to accommodate new business goals, or to keep in step with changes in the supervised learning technology, wherein if it is determined to change the categorization scheme, steps (f) through (i) are repeated.

4. A method as recited in claim 1, wherein the step of analyzing the business domain further comprises steps:

analyzing anticipated content of relevant electronic communications;

analyzing business missions and goals;

evaluating skills of involved personnel;

analyzing the organization'"'"'s workflow;

analyzing use of stored responses including determining whether answers have been developed for frequently occurring questions; and

producing business requirements for use in the validation step using insight gained by the analysis of the business domain.

5. A method as recited in claim 4, wherein the step of analyzing business missions and goals further comprises:

reviewing a model of the business domain and determining success criteria and measurements used to determine when the business is successful;

establishing turnaround times for the electronic communications to support mission and goals of the business; and

determining a volume of electronic communications received daily and determining a number of received communications that must be answered to meet the goals of the business.

6. A method as recited in claim 4, wherein the step of analyzing the organization'"'"'s workflow, further comprises:

determining a workflow through the organization and routing performed on a category by category basis;

determining if subject matter experts (SME) have been established for categories of information; and

determining whether an automated or manual system for routing electronic communications is being used.

7. A method as recited in claims 1, wherein the step of defining a categorization scheme further comprises steps:

combining lists of categories in the group of categories related to business mission groups, related to routing communications to specific individuals, communications for which an automated response is feasible and desirable, and those related to existing stored responses or stored templates for responses;

determining technically feasible categorization from the assembled categorization scheme;

correlating knowledge of the technical structure of the communications with knowledge of what kinds of features can actually be identified by the machine learning component; and

eliminating or combining categories with few examples, if necessary.

8. A computer-implemented method for categorizing incoming electronic communications using a supervised machine learning component, and for factoring an organization'"'"'s business domain the technology domain to enable an acceptable automated response and routing scheme, said method comprising the steps of:

selecting a machine learning component for the technology domain;

preparing a set of training data comprising representations of previously categorized electronic communications, wherein the data in an electronic communication is textual and each electronic communication has features, where a feature is related to textual data;

analyzing the organization'"'"'s business domain with respect to desired routing and handling of contemplated message categories of electronic communications, the analysis resulting in identification of tasks to be performed and actions to be taken in response to a received electronic communication of a contemplated message category, the analysis also resulting in identification of features relevant to categorization of electronic communications;

determining skill levels of personnel corresponding to required tasks and actions identified in the step of analyzing the organization'"'"'s business domain;

extracting, using a computer, a new representation of each electronic communication in the training set depending on a frequency of occurrence in the electronic communication of features identified as relevant to the business domain;

inducing a pattern characterization when an electronic communication belongs to a category, wherein the patterns are presented as rules or another format correspond the selected machine learning component; and

developing, using said computer, an initial categorization scheme based on areas of the business domain receiving a greater quantity of electronic communications or electronic communications of a relatively higher priority.

9. A method as recited in claim 8, wherein an electronic communication comprises more than one part and each part of the electronic communication has corresponding features related to a category and categorized based on each part in the inducing step.