Effecting action to address an issue associated with a category based on information that enables ranking of categories

• US 7,593,904 B1
• Filed: 06/30/2005
• Issued: 09/22/2009
• Est. Priority Date: 06/30/2005
• Status: Expired due to Fees

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Tripp, Thomas M., Kirshenbaum, Evan R., Suermondt, Henri J., Forman, George H.
• Primary Examiner(s): Holmes; Michael B
• Application Number:
  US11/172,187
• Time in Patent Office: 1,545 Days
• Field of Search: 706/12
• US Class Current: 706/12
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A categorizer is trained for plural categories according to a machine-learning algorithm. The categorizer classifies cases in a set of cases into the plural categories. One or more quantification measures regarding cases in the data set are computed based on output from the categorizer, and information is provided to enable ranking of the categories based on the one or more quantification measures. Action is effected to address an issue associated with at least one of the categories based on the provided information.

Claims

1. A method comprising:

training a categorizer for plural categories using a machine-learning algorithm;

classifying, by the categorizer, a set of cases into the plural categories;

computing one or more quantification measures regarding cases in the set of cases based on output from the categorizer;

providing information to enable ranking of the categories based on the one or more quantification measures; and

effecting action to address an issue associated with at least one of the categories based on the provided information,

wherein at least the classifying, computing, and providing are performed by software stored on one or more machine-readable media and executed by at least one processor.

2. The method of claim 1, wherein the set of cases comprises a first set of cases, wherein training the categorizer comprises:

identifying cases from a second set of cases based on a query;

receiving confirmation or disconfirmation as to whether cases from the identified cases are examples of a particular one of the categories;

identifying training cases for the particular one of the categories based on the confirmation or disconfirmation, the training cases for training the categorizer.

3. The method of claim 2, wherein identifying the plural categories comprises running a clustering algorithm on cases from the second set of cases.

4. The method of claim 1, wherein classifying the cases into plural categories comprises classifying the cases for plural problems encountered by customers.

5. The method of claim 1, wherein the set of cases comprises a first set of cases, and wherein training the categorizer is based on a second set of cases, the method further comprising receiving the second set of cases based on receiving data by one or more of:

recording the data by support personnel;

generating a transcript of a telephone conversation;

recording a telephone conversation;

receiving an e-mail message;

providing a record of pages visited on a web site;

providing an output from a diagnostic program;

storing the data in a database;

receiving a computer file from a customer;

receiving the data from a survey; and

receiving the data by a monitoring device.

6. The method of claim 5, wherein recording the data by support personnel comprises one or more of recording a description of an interaction between the support personnel and a customer;

recording selection of a category by the support personnel; and

recording an indication of a state of mind of a customer.

7. The method of claim 5, wherein providing a record of the pages visited by a web site comprises one or more of recording a document viewed by a customer and recording a document viewed by support personnel.

8. The method of claim 5, wherein storing the data in the database comprises one or more of storing information relating to a prior call;

storing information relating a purchase; and

storing demographic information relating to a customer.

9. The method of claim 1, wherein training the categorizer using the machine-learning algorithm comprises training the categorizer using one of a support vector machine learning algorithm;

naï

ve Bayes network learning algorithm;

a Bayesian network learning algorithm;

a neural network learning algorithm; and

a decision tree learning algorithm.

10. The method of claim 1, wherein computing the one or more quantification measures comprises computing one or more of:

sum of duration of calls represented by the cases;

an amount of money to resolve the cases;

amount of time to resolve the cases;

amount of lost revenue due to the cases;

a degree of customer aggravation;

an amount of time spent by customers before initiating a call;

an amount of time spent by customers during the call;

an amount of time spent diagnosing cases;

amount of money spent by customers;

an amount of money spent interacting with customers;

an amount of money spent diagnosing the cases; and

a number of customers who declined to initiate a call.

11. The method of claim 1, wherein providing the information comprises providing information ranking the categories according to at least one or more of:

number of cases in the categories;

amount of time involved in resolving cases in the categories;

amount of money spent to resolve cases in the categories;

amount indicating customer dissatisfaction due to the cases in the categories;

amount of lost revenue due to the cases in the categories; and

amount indicating lost customers due to the cases in the categories.

12. The method of claim 1, further comprising ranking the categories based on the provided information.

13. The method of claim 12, further comprising producing a report to provide information regarding the ranked categories, wherein producing the report comprises producing a report containing user-selectable fields to obtain further information regarding at least one of:

details regarding the ranked categories, and subcategories of the ranked categories.

14. The method of claim 13, wherein producing the report comprises producing an interactive document to allow a user to change one or more criteria regarding ranking of the categories.

15. The method of claim 1, wherein the categories are part of a hierarchy of categories, the method further comprising receiving user input to add one or more additional categories to the hierarchy.

16. The method of claim 1, further comprising employing a clustering algorithm to cluster cases to identify example cases for the categories.

17. The method of claim 1, further comprising adjusting the one or more quantification measures based on one or more factors, wherein providing information to rank the categories is based on the adjusted one or more quantification measures.

18. The method of claim 17, wherein adjusting the one or more quantification measures is based on information about the categorizer, the information comprising one of (1) a quality measure relating to the performance of the categorizer in classifying training cases for a particular category;

and (2) a statistical distribution of scores computed by the categorizer for classifying training cases for the particular problem.

19. The method of claim 17, wherein adjusting the one or more quantification measures comprises extrapolating the one or more quantification measures for a future time period.

20. The method of claim 17, wherein adjusting the one or more quantification measures comprises normalizing the one or more quantification measures based on the one or more factors.

21. The method of claim 20, wherein normalizing the one or more quantification measures is based on at least one of sales numbers and trends, marketing promotions, seasonal buying patterns, product end-of-life information, and product introduction information.

41. The method of claim 1, wherein the training, classifying, computing, and providing are performed by the at least one processor in a customer support system.

42. The method of claim 1, wherein the training, classifying, computing, and providing are performed by a call agent system that includes call agent stations and at least one analysis server.

43. The method of claim 2, wherein identifying the training cases comprises adding at least one of the training cases to a positive training set in response to a confirmation and at least another of the training cases to a negative training set in response to a disconfirmation,

wherein the categorizer is trained using the positive and negative training sets.

22. A method comprising:

training a categorizer for plural problems associated with a customer support organization based on a machine-learning algorithm;

classifying, by the categorize, incidents in a first data set into the plural problems;

computing one or more quantification measures regarding the plural problems based on output from the categorizer; and

presenting information relating to relative magnitude of the problems based on the one or more quantification measures; and

taking action in the customer support organization to remedy at least one of the problems based on the presented information,

wherein at least the classifying, computing, and presenting are performed by software stored on one or more machine-readable media and executed by at least one processor.

23. The method of claim 22, wherein presenting the information comprises presenting information of at least one of the following problems:

(1) a predefined number of problems associated with highest quantification measures;

(2) problems associated with highest quantification measures that when aggregated exceeds a predefined threshold; and

(3) problems associated with quantification measures exceeding a predefined threshold.

24. The method of claim 22, wherein presenting the information comprises presenting information to identify higher priority problems in a predetermined frame of reference.

25. The method of claim 24, wherein presenting the information to identify the higher priority problems in the predetermined frame of reference comprises presenting the information to identify the higher priority problems in at least one of a predetermined time period, a product line, and a geographic region.

26. The method of claim 22, further comprising receiving information associated with the incidents in the first data set from one or more call agent stations.

27. The method of claim 22, wherein computing the one or more quantification measures is performed by a quantifier, the method further comprising computing one or more other quantification measures, by the quantifier, associated with incidents in a second data set based on output from the categorizer;

and

presenting information to identify higher priority problems for the second data set based on the one or more other quantification measures for the second data set.

28. The method of claim 27, further comprising displaying outputs regarding the information identifying higher priority problems for the first data set and the information identifying higher priority problems for the second data set to enable comparison of the outputs.

29. The method of claim 28, wherein the first and second data sets correspond to different time periods, and wherein displaying the outputs comprises displaying outputs regarding information identifying higher priority problems for data sets of different time periods.

30. The method of claim 22, wherein presenting the information comprises producing a document containing information indicated by one or more of the following parameters:

a selection criterion for problems to display;

a desired depth to display in a hierarchy of problems;

an ordering criterion used to rank problems;

a problem to focus on;

a set of information related to a problem to display;

a selection of a presentation format; and

a selection of a data set to use as the first data set.

31. The method of claim 30, wherein producing the document comprises producing an interactive document providing a user with an ability to dynamically alter on or more of the parameters.

32. The method of claim 30, wherein presenting the information comprises producing a report containing information associated with a subset of the first data set, the subset based on one or more of a particular product, a geographic region, demographic information of customers, date of incidents, purchase date, production introduction date, and product manufacture date.

33. The method of claim 22, wherein taking the action comprises one or more of:

allocating a sum of money;

defining a budget;

allocating a physical resource;

hiring a person;

assigning a task to a person;

writing a document;

modifying an existing document;

identifying a document;

altering availability of a document;

altering an organization of a web site;

modifying a design of a product;

modifying a packaging of a product;

modifying a manufacturing process for a product;

creating a software program;

modifying a software program;

creating a patch for a software program;

contacting a customer, vendor, supplier, employee, or partner;

modifying a marketing campaign;

changing response time of service providers;

training service personnel;

discontinuing efforts that are no longer required;

changing the process of writing and delivery of software programs;

taking actions with reference to seasonal fluctuations; and

providing reports to customers regarding how issues are being monitored and addressed.

34. The method of claim 22, further comprising repeating the classifying, computing, presenting, and taking action as additional incidents are received.

35. The method of claim 22, further comprising:

inspecting classifications of a subset of the incidents in the first data set performed by the categorizer;

computing a measure of classifications determined to be either incorrect or for which the categorizer is insufficiently confident about; and

retraining the categorizer in response to the measure exceeding a predefined threshold.

44. The method of claim 22, wherein the training, classifying, computing, and presenting are performed by the at least one processor in a customer support system.

36. A machine-readable storage medium containing instructions that upon execution by at least one processor cause a system to:

receive queries relating to respective plural customer support issues;

identify cases from a data set in response to the queries;

receive first indications that a first portion of the identified cases belong to the customer support issues, and second indications that a second portion of the identified cases do not belong to the customer support issues;

train a categorizer for the plural customer support issues based on the received first and second indications;

classify, by the categorizer, cases in a data set into the plural customer support issues;

quantify the cases in the data set based on the classifying by the categorizer; and

produce information to enable ranking of the customer support issues based on the quantifying.

45. The machine-readable storage medium of claim 36, wherein the instructions when executed cause the system to add the first portion of the identified cases to a positive training set and the second portion of the identified cases to a negative training set, wherein the categorizer is trained based on the positive and negative training sets.

37. A system comprising:

a storage to store a data set having plural cases;

a categorizer;

a search engine to identify cases from the data set in response to queries relating to plural customer support issues;

an interface to receive indications of whether the identified cases belong to the plural customer support issues;

a training module to train the categorizer based on the received indications, wherein the categorizer is adapted to classify the cases into the plural customer support issues;

a quantifier to compute one or more quantification measures for the cases in the plural customer support issues based on output from the categorizer; and

a ranking module to provide information to prioritize the customer support issues based on the one or more quantification measures.

38. The system of claim 37, further comprising an adjustment module to adjust the one or more quantification measures, the ranking module to provide information to prioritize the customer support issues based on adjusted one or more quantification measures.

39. The system of claim 38, the adjustment module to adjust the one or more quantification measures by performing one of:

extrapolating the one or more quantification measures for a future time period; and

normalizing the one or more quantification measures based on one or more factors.

40. An article comprising a machine-readable storage medium containing instructions that upon execution by at least one processor cause a system to:

receive queries relating to respective plural problems;

identify incidents from a first data set in response to the queries;

receive indications regarding whether the identified incidents belong to the problems;

train a categorizer for the plural problems based on the received indications;

classify, by the categorizer, incidents from a second data set into the plural problems;

compute one or more quantification measures for the incidents for the plural problems based on output from the categorizer; and

present information to prioritize the problems based on the one or more quantification measures; and

effect action to address at least one of the problems based on the information prioritizing the problems.