Computing a quantification measure associated with cases in a category

• US 9,047,290 B1
• Filed: 04/29/2005
• Issued: 06/02/2015
• Est. Priority Date: 04/29/2005
• Status: Expired due to Fees

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Kirshenbaum, Evan R., Suermondt, Henri J., Forman, George H.
• Primary Examiner(s): Saeed, Usmaan
• Assistant Examiner(s): VO, CECILE H
• Application Number:
  US11/118,786
• Time in Patent Office: 3,686 Days
• Field of Search: 715/500, 715/719, 707/1, 707/5, 707/6, 707/102, 707/2, 707/1/1, 707/3, 707/100, 707/101, 707/651, 707/710, 707/737, 707/E17.09, 707/E17.89, 707/705, 707/723, 707/740, 707/748, 707/999.003, 382/109, 382/173, 382/228, 382/190, 382/159, 382/155, 382/224, 706/12, 706/20, 705/7.32
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06N 5/00 Computing arrangements using knowledge-based models

Abstract

At least one case from a set of cases is identified in response to a query relating to at least one category. An indication is received regarding whether the at least one case belongs to the category. A categorizer is trained based on the received indication. A quantification measure associated with cases in the category is computed based on output from the categorizer.

Claims

1. A method comprising:

identifying, in response to a query relating to at least one category, plural cases from a set of cases;

receiving, by a system having a processor, indications corresponding to user confirmation or user disconfirmation of individual ones of the plural cases as belonging to the category, wherein user confirmation of a first of the plural cases as belonging to the category includes user selection of a first user-selectable element in a display screen, and wherein user disconfirmation of a second of the plural cases as belonging to the category includes user selection of a second user-selectable element in the display screen;

in response to the indications corresponding to the user confirmation of the first case and the user disconfirmation of the second case, adding, by the system, the first case to a positive training set, and the second case to a negative training set;

training, by the system, a categorizer based on the positive and negative training sets;

computing, by the system, a quantification measure associated with cases in the category based on output from the trained categorizer, wherein each of at least some of the cases in the category has a data field, and wherein computing the quantification measure comprises computing an aggregate of the data fields of the at least some of the cases in the category; and

adjusting, by the system, the quantification measure to calibrate for inaccuracy of the trained categorizer.

2. The method of claim 1, wherein computing the quantification measure comprises computing an estimated quantification measure.

3. The method of claim 1, wherein the category is part of a hierarchy of categories, the method further comprising:

in response to the query, displaying summaries of a subset of the cases from the set; and

receiving an input by a category editor to add at least one additional category to the hierarchy, the input based on the displayed summaries.

4. The method of claim 3, further comprising:

computing, based on output from the trained categorizer, at least another quantification measure for at least another category in the hierarchy.

5. The method of claim 4, further comprising listing the categories of the hierarchy in an order according to the quantification measures.

6. The method of claim 1, further comprising:

receiving an additional set of cases;

computing a quantification measure associated with the additional set of cases; and

repeating the receiving of the additional set of cases and computing the quantification measure associated with the additional set of cases at plural intervals.

7. The method of claim 6, wherein repeating the receiving and computing at the plural intervals comprises repeating the receiving and computing at intervals of less than one of:

a year, a month, a week, a day, a minute, and a second.

8. The method of claim 1, wherein computing the aggregate comprises computing at least one of a sum, an average, a standard deviation, a percentage level confidence interval, a variance, a minimum, a maximum, a median, a mode, a geometric mean, a harmonic mean, a percentile rank, and an ordinal statistic.

9. The method of claim 1,

wherein adjusting the quantification measure is based on using a statistical distribution of values for the data fields associated both with cases indicated as being in the category by the categorizer and cases indicated as not being in the category by the categorizer.

10. The method of claim 1, wherein computing the quantification measure is performed by a quantifier, the method further comprising:

calibrating the quantifier to adjust the quantification measure.

11. The method of claim 1, further comprising:

obtaining from the trained categorizer a score associated with each of a group of cases;

computing a statistical distribution based on the scores,

wherein adjusting the quantification measure is based on the statistical distribution.

12. The method of claim 1, wherein the set of cases comprises a first set of cases, the method further comprising:

computing another quantification measure associated with cases in a second set that have been categorized by the categorizer as belonging to the category; and

generating a report presenting the quantification measure associated with the first set of cases and the quantification measure associated with the second set of cases.

13. The method of claim 1, wherein computing the quantification measure comprises computing the quantification measure with respect to cases of a first partition of the set, the method further comprising:

computing another quantification measure with respect to cases of a second partition of the set; and

generating a report that presents the quantification measures.

14. The method of claim 13, further comprising partitioning the cases of the set into the first partition and the second partition based on data fields associated with the cases.

15. The method of claim 1, further comprising creating a quantifier that is associated with the trained categorizer, the quantifier computing the quantification measure based on information from the trained categorizer.

16. The method of claim 15, further comprising using the quantifier and the trained categorizer to categorize and quantify cases for a second set of cases.

17. The method of claim 1, further comprising creating a quantifier that includes the trained categorizer, the quantifier computing the quantification measure based on information from the trained categorizer.

18. The method of claim 1, further comprising:

displaying the quantification measure as a running estimate; and

updating the running estimate based on further training of the categorizer.

19. A system comprising:

at least one processor;

a data set comprising cases;

a search engine to identify plural cases from the data set in response to a query relating to at least one category;

a confirmation module executable on the at least one processor to receive indications corresponding to user confirmation or user disconfirmation of individual ones of the plural cases as belonging to the category, wherein user confirmation of a first of the plural cases as belonging to the category includes user selection of a first user-selectable element in a display screen, and wherein user disconfirmation of a second of the plural cases as belonging to the category includes user selection of a second user-selectable element in the display screen, the confirmation module to further, in response to the indications corresponding to the user confirmation of the first case and the user disconfirmation of the second case, add the first case to a positive training set, and the second case to a negative training set;

a categorizer to be trained based on the positive training set and negative training set; and

a quantifier to compute a quantification measure associated with cases in the category based on output from the trained categorizer, wherein each of the cases in the category has a data field, and the quantification measure is computed by aggregating the data fields of the cases in the category, and wherein the quantifier is to adjust, to calibrate for inaccuracy of the trained categorizer, the quantification measure based on a measure of accuracy of the trained categorizer.

20. The system of claim 19, wherein the search engine, the confirmation module, the categorizer, and the quantifier are executable concurrently to perform respective tasks.

21. The system of claim 19, wherein the categorizer is to classify the cases in the data set into the category.

22. The system of claim 21, wherein the categorizer is to provide information regarding the classification of the cases to the quantifier.

23. The system of claim 19, wherein the category is in a hierarchy of categories, and wherein the search engine is to cause display of summaries of a subset of the plural cases in response to the query, the system further comprising:

a category editor to add a new category to the hierarchy, in response to input based on the displayed summaries.

24. An article comprising at least one non-transitory storage medium containing instructions that when executed cause a system to:

identify, in response to a query relating to at least one category, plural cases from a set of cases;

receive indications corresponding to user confirmation or user disconfirmation of individual ones of the plural cases as belonging to the category, wherein user confirmation of a first of the plural cases as belonging to the category includes user selection of a first user-selectable element in a display screen, and wherein user disconfirmation of a second of the plural cases as belonging to the category includes user selection of a second user-selectable element in the display screen;

in response to the indications corresponding to the user confirmation of the first case and the user disconfirmation of the second case, adding the first case to a positive training set, and the second case to a negative training set;

train a categorizer based on the positive and negative training sets;

compute a quantification measure associated with cases in the category based on output from the trained categorizer, wherein each of the cases in the category has a data field, and wherein computing the quantification measure comprises computing an aggregate of the data fields of the cases in the category; and

adjust, to calibrate for inaccuracy of the trained categorizer, the quantification measure based on a measure of accuracy of the trained categorizer.