Classification confidence estimating tool

• US 8,429,098 B1
• Filed: 04/30/2010
• Issued: 04/23/2013
• Est. Priority Date: 04/30/2010
• Status: Active Grant

• Current Assignee: Global E Procure Incorporated
• Original Assignee: Global E Procure Incorporated
• Inventors: Pawar, Sachin Sharad, Joshi, Girish, Katakol, Santosh
• Primary Examiner(s): Vincent, David
• Assistant Examiner(s): Chubb, Mikayla
• Application Number:
  US12/799,725
• Time in Patent Office: 1,089 Days
• Field of Search: None
• US Class Current: 706/12
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/353 into predefined classes

Abstract

Disclosed is a tool that assesses and classifies data in a data set to and compares those assessments with nominal attributes and text attributes of data present in a new record to assign a classification to the new record. The classification assigned to the new record is provided a confidence level based on both a qualitative factor and a quantitative factor. The qualitative factor may be calculated by forming of a list of important words for each class and comparing the list to data in the new record and converting the comparison into a confidence level; the quantitative factor may be calculated by estimating the importance or weight of several factors, ratios of certain probabilities related to the most likely class and to the second most likely class, and using the importance of the factors and matchfactors to scale the resulting ratio, then transforming the resulting ratio into a confidence level.

Claims

1. A non-transitory computer readable medium for estimating a confidence level of a classification assignment, the non-transitory computer readable medium having instructions configured to cause a processor of a computer to:

create a data model having a plurality of classes of records, the data model being based on a set of pre-classified records;

assign one of said classes to a new record;

obtain a qualitative confidence level for the assignment of said class;

obtain a quantitative confidence level for the assignment of said class; and

combine the qualitative confidence level and the quantitative confidence level, wherein the quantitative confidence level is obtained by finding a ratio of the probability of said class to a second most likely class, finding a ratio of a complement probability of said second most likely class to a complement probability of said class, and using matchfactors to generate a scaled ratio.

2. The non-transitory computer readable medium claimed in claim 1 wherein the non-transitory computer readable medium is configured to devise from said data model a list of important words for each class to obtain the qualitative confidence level.

3. The non-transitory computer readable medium claimed in claim 1 wherein the non-transitory computer readable medium is configured to estimate weights for each of a plurality of factors effecting said quantitative confidence level to obtain the quantitative confidence level.

4. The non-transitory computer readable medium claimed in claim 1 wherein the non-transitory computer readable medium is configured to devise a list of important words for each of said plurality of classes, selecting a set of at least one condition evidentiary of said class, and determining if new record meets said set of at least one conditions.

5. The non-transitory computer readable medium claimed in claim 1 wherein the non-transitory computer readable medium is configured to transform said scaled ratio to a reliability value and comparing said reliability value with a pre-determined threshold for confidence level to obtain the quantitative confidence level.

6. The non-transitory computer readable medium claimed in claim 4 wherein the non-transitory computer readable medium is configured to determine a mutual information score for each possible combination of one of a list of words and one of said plurality of classes to devise the list of important words for each of said plurality of classes.

7. A non-transitory computer readable medium for estimating a confidence level of a classification assignment, the non-transitory computer readable medium comprising a set of instructions stored on a computer readable medium which, when read by a processor, cause a computer to:

create a data model having a plurality of classes of records, the data model being based on a set of pre-classified records,

assign one of said classes to a new record,

obtain a qualitative confidence level for the assignment of said class,

obtain a quantitative confidence level for the assignment of said class, and

combine the qualitative confidence level and the quantitative confidence, wherein the non-transitory computer readable medium is configured to estimate a weight for each of a plurality of factors affecting said quantitative confidence level, find a ratio of the probability of said class to the probability of second most likely class, find a ratio of a complement probability of said second most likely class to a complement probability of said class, and use matchfactors to generate a scaled ratio.

8. The non-transitory computer readable medium as claimed in claim 7, wherein the non-transitory computer readable medium is configured to generate a qualitative confidence level and evaluate the qualitative confidence level.

9. The non-transitory computer readable medium as claimed in claim 8 wherein the non-transitory computer readable medium is further configured to devise from said data model a list of important words for each class including listing words belonging to the class and determine a mutual information score for each possible combination of one of the list of words and one of said plurality of classes, select a set of at least one condition evidentiary of said class, and determine if new record meets said set of at least one conditions to generate the qualitative confidence level.

10. The non-transitory computer readable medium as claimed in claim 7 wherein the non-transitory computer readable medium is further configured to devise from said data model a list of important words for each class including listing words belonging to the class and determine a mutual information score for each possible combination of one of the list of words and one of said plurality of classes, select a set of at least one condition evidentiary of said class, and determine if said new record meets said set of at least one conditions to generate the qualitative confidence level.

11. A non-transitory computer readable medium for estimating the confidence level of a classification assignment comprising, the non-transitory computer readable medium including instructions configured to cause a processor of a computer to:

a) create a data model from a set of pre-classified records, the data model having a plurality of classes of records;

b) assign one of said classes to a new record;

c) obtain a qualitative confidence level for the assignment of said class comprising finding a list of words belonging to the class;

filtering said list to eliminate any noise words, and for each word on said list, finding a Mutual Information Score for each possible combination of one of the list of words and one of said plurality of classes using the following;

12. A non-transitory computer readable medium for estimating the confidence level of a classification assignment comprising, the non-transitory computer readable medium being configured to cause a processor of a computer to:

a) create a data model from a set of pre-classified, the data model having a plurality of classes of records;

b) assign one of said classes to a new record;

c) obtain a qualitative confidence level for the assignment of said class comprising finding a list of words belonging to the class;

d) filter said list to eliminate any noise words, and for each word on said list, find a Mutual Information Score for each possible combination of one of the list of words and one of said plurality of classes;

e) obtain a quantitative confidence level for the assignment of said class using the following;

determining a ratio of probabilities of a most likely class to a second most likely class;

13. A non-transitory computer readable medium for estimating the confidence level of a classification assignment comprising, the non-transitory computer readable medium being configured to cause a processor of a computer to:

a) create a data model from a set of pre-classified records, the data model having a plurality of classes of records;

b) assign one of said classes to a new record;

c) obtain a qualitative confidence level for the assignment of said class comprising finding a list of words belonging to the class;

d) filter said list to eliminate any noise words, and for each word on said list, find a Mutual Information Score for each possible combination of one of the list of words and one of said plurality of classes using the following;