CLASSIFICATION OF DATA

• US 20100179936A1
• Filed: 01/09/2009
• Published: 07/15/2010
• Est. Priority Date: 01/09/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Jeremiah, Garreth
• Granted Patent: US 8,311,969 B2
• Time in Patent Office: Days
• US Class Current: 706/50
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/24 Classification techniques

Abstract

A method and system for reducing false positives in the classification of data is provided, wherein the data can be categorized into fields, including creating an assertion table or assessing an existing assertion table for the data whereby the data is placed into categories and each category is assigned one or more classifications, setting a positive and/or negative assertion ratio for each category, determining the accuracy of each classification by assessing a percentage of the data in each category to see if the data is correctly identified by the classification, if the positive assertion ratio is reached, maintaining the classification for each category of data, if the negative assertion ratio is reached, de-asserting the classification.

Claims

1. A method of reducing false positives in the classification of data, wherein the data can be categorized into columns, comprising:

creating an assertion table or assessing an existing assertion table for the data whereby the data is placed into categories and each category is assigned one or more classifications;

setting a positive and/or negative assertion ratio for each category;

determining the accuracy of each classification by assessing a percentage of the data in each category to see if the data is correctly identified by the classification;

if the positive assertion ratio is reached, maintaining the classification for each category of data;

if the negative assertion ratio is reached, de-asserting the classification.

2. The method of claim 1 wherein the positive assertion ratio comprises the percentage of data that must match the classification to maintain the classification and the negative assertion ratio comprises the percentage of data that does not match the classification.

3. The method of claim 1 wherein a positive assertion comprises data that is identified accurately by the classification and a negative assertion comprises data that is not identified accurately by the classification.

4. The method of claim 3 wherein data that is neither identified as positive or negative is not counted as a positive assertion or a negative assertion.

5. The method of claim 3 wherein the data that is neither identified as positive or negative is counted as either a positive assertion or a negative assertion.

6. The method of claim 1 wherein the percentage of the data that is assessed is less than one hundred percent.

7. The method of claim 1 wherein the percentage of the data that is assessed is one hundred percent.

8. The method of claim 1 wherein the data is already identified with a classification prior to creating the assertion table.

9. The method of claim 1 wherein the data is not identified with a classification prior to creating the assertion table.

10. The method of claim 1 wherein setting the positive and negative assertion ratios is dependent upon the type of category.

11. The method of claim 1 wherein the categories comprise data that is personal to an entity.

12. The method of claim 1 wherein the categories comprise social security numbers, drivers licenses, names, birthdates, telephone numbers, email addresses, insurance numbers, business account numbers, account information, credit card numbers, bank account numbers, health identifiers, addresses, dates, currency values, health related records, regulatory identified data and national identification numbers.

13. A system for reducing false positives in the classification of data comprising:

at least one database having one or more files of data, wherein the data can be categorized into fields;

a user interface linked to the database; and

a validation manager for creating an assertion table or assessing an existing assertion table for the data whereby the data is placed into categories and each category is assigned one or more classifications;

setting a positive and/or negative assertion ratio for each category;

determining the accuracy of each classification by assessing a percentage of the data in each category to see if the data is correctly identified by the classification;

if the positive assertion ratio is reached, maintaining the classification for each category of data;

if the negative assertion ratio is reached, de-asserting the classification.

14. The system of claim 13 wherein the positive assertion ratio comprises the percentage of data that must match the classification to maintain the classification and the negative assertion ratio comprises the percentage of data that does not match the classification.

15. The system of claim 13 wherein the percentage of the data that is assessed is less than one hundred percent.

16. The system of claim 13 wherein the percentage of the data that is assessed is one hundred percent.

17. A computer program product encoded in a computer readable medium for instructing a system to reducing false positives in the classification of data, wherein the data can be categorized into fields, the program code configured to cause the computer to perform the method comprising:

creating an assertion table or assessing an existing assertion table for the data whereby the data is placed into categories and each category is assigned one or more classifications;

setting a positive and/or negative assertion ratio for each category;

determining the accuracy of each classification by assessing a percentage of the data in each category to see if the data is correctly identified by the classification;

if the positive assertion ratio is reached, maintaining the classification for each category of data;

if the negative assertion ratio is reached, de-asserting the classification.

18. The computer program product of claim 17 wherein the positive assertion ratio comprises the percentage of data that must match the classification to maintain the classification and the negative assertion ratio comprises the percentage of data that does not match the classification.

19. The computer program product of claim 17 wherein the percentage of the data that is assessed is less than one hundred percent.

20. The computer program product of claim 17 wherein the percentage of the data that is assessed is one hundred percent.