Data classification methods using machine learning techniques

• US 7,958,067 B2
• Filed: 05/23/2007
• Issued: 06/07/2011
• Est. Priority Date: 07/12/2006
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Sarah, Anthony, Borrey, Roland, Schmidtler, Mauritius A. R.
• Primary Examiner(s): Fernandez Rivas; Omar F
• Application Number:
  US11/752,673
• Publication Number:
  US 20080082352A1
• Time in Patent Office: 1,476 Days
• Field of Search: 706/12, 706/15, 706/20, 706/21, 706/45, 706/52, 706/62
• US Class Current: 706/20
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 50/18 Legal services
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Methods for classifying documents are presented. Methods for analyzing documents associated with legal discovery are also presented. Methods for cleaning up data are also presented. Methods for verifying an association of an invoice with an entity are also presented. Method for managing medical records are presented. Method for face recognition are presented.

Claims

1. A method for classifying documents, comprising:

receiving at least one labeled seed document having a known confidence level of label assignment;

receiving unlabeled documents;

receiving at least one predetermined cost factor;

training a transductive classifier through iterative calculation using the at least one predetermined cost factor, the at least one seed document, and the unlabeled documents, wherein for each iteration of the calculations the cost factor is adjusted as a function of an expected label value;

after at least some of the iterations, storing confidence scores for the unlabeled documents; and

outputting identifiers of the unlabeled documents having the highest confidence scores to at least one of a user, another system, and another process.

2. The method of claim 1, wherein the at least one seed document has a list of keywords.

3. The method of claim 1, wherein confidence scores are stored after each of the iterations, wherein an identifier of the unlabeled document having the highest confidence score after each iteration is output to the at least one of a user, another system, and another process.

4. The method of claim 1, further comprising receiving a data point label prior probability for the labeled and unlabeled documents, wherein for each iteration of the calculations the data point label prior probability is adjusted according to an estimate of a data point class membership probability.

5. A method for analyzing documents associated with legal discovery, comprising:

receiving documents associated with a legal matter;

performing a document classification technique on the documents using a transductive classifier trained through iterative calculation using at least one predetermined cost factor, and at least one seed document, wherein for each iteration of the calculations the cost factor is adjusted as a function of an expected label value; and

outputting identifiers of at least some of the documents based on the classification thereof.

6. The method of claim 5, wherein the document classification technique includes a transductive process.

7. The method of claim 6, further comprising training a transductive classifier through iterative calculation using at least one predetermined cost factor, at least one seed document, and the documents associated with the legal matter, wherein for each iteration of the calculations the cost factor is adjusted as a function of an expected label value, and using the trained classifier to classify the received documents.

8. The method of claim 7, further comprising receiving a data point label prior probability for labeled and unlabeled documents, wherein for each iteration of the calculations the data point label prior probability is adjusted according to an estimate of a data point class membership probability.

9. The method of claim 5, wherein the document classification technique includes a support vector machine process.

10. The method of claim 5, wherein the document classification technique includes a maximum entropy discrimination process.

11. The method of claim 5, further comprising outputting a representation of links between the documents.

12. A method for cleaning up data, comprising:

receiving a plurality of labeled data items;

selecting subsets of the data items for each of a plurality of categories;

setting an uncertainty for the data items in each subset to about zero;

setting an uncertainty for the data items not in the subsets to a predefined value that is not about zero;

training a transductive classifier through iterative calculation using the uncertainties, the data items in the subsets, and the data items not in the subsets as training examples;

applying the trained classifier to each of the labeled data items to classify each of the data items; and

outputting a classification of the input data items, or derivative thereof, to at least one of a user, another system, and another process.

13. The method of claim 12, wherein the subsets are selected at random.

14. The method of claim 12, wherein the subsets are selected and verified by a user.

15. The method of claim 12, further comprising changing the label of at least some of the data items based on the classification.

16. The method of claim 12, wherein identifiers of data items having a confidence level below a predefined threshold after classification thereof are output to a user.

17. A method for verifying an association of an invoice with an entity, comprising;

training a classifier based on an invoice format associated with a first entity, wherein the classifier is a transductive classifier, and further comprising training the transductive classifier through iterative calculation using at least one predetermined cost factor, at least one seed document, and the invoices, wherein for each iteration of the calculations the cost factor is adjusted as a function of an expected label value;

accessing a plurality of invoices labeled as being associated with at least one of the first entity and other entities;

performing a document classification technique on the invoices using the classifier; and

outputting an identifier of at least one of the invoices having a high probability of not being associated with the first entity.

18. The method of claim 17, wherein the document classification technique includes a transductive process.

19. The method of claim 17, further comprising receiving a data point label prior probability for the seed document and invoices, wherein for each iteration of the calculations the data point label prior probability is adjusted according to an estimate of a data point class membership probability.

20. The method of claim 17, wherein the document classification technique includes a support vector machine process.

21. The method of claim 17, wherein the document classification technique includes a maximum entropy discrimination process.

22. A method for managing medical records, comprising:

training a classifier based on a medical diagnosis, wherein the classifier is a transductive classifier, and further comprising training the transductive classifier through iterative calculation using at least one predetermined cost factor, at least one seed document, and the medical records, wherein for each iteration of the calculations the cost factor is adjusted as a function of an expected label value;

accessing a plurality of medical records;

performing a document classification technique on the medical records using the classifier; and

outputting an identifier of at least one of the medical records having a low probability of being associated with the medical diagnosis.

23. The method of claim 22, wherein the document classification technique includes a transductive process.

24. The method of claim 22, further comprising receiving a data point label prior probability for the seed document and medical records, wherein for each iteration of the calculations the data point label prior probability is adjusted according to an estimate of a data point class membership probability.

25. The method of claim 22, wherein the document classification technique includes a support vector machine process.

26. The method of claim 22, wherein the document classification technique includes a maximum entropy discrimination process.

27. A method for face recognition, comprising:

receiving at least one labeled seed image of a face, the seed image having a known confidence level;

receiving unlabeled images;

receiving at least one predetermined cost factor;

training a transductive classifier through iterative calculation using the at least one predetermined cost factor, the at least one seed image, and the unlabeled images, wherein for each iteration of the calculations the cost factor is adjusted as a function of an expected label value;

after at least some of the iterations, storing confidence scores for the unlabeled seed images; and

outputting identifiers of the unlabeled images having the highest confidence scores to at least one of a user, another system, and another process.

28. The method of claim 27, wherein the at least one seed image has a label indicative of whether the image is included in a designated category.

29. The method of claim 27, wherein confidence scores are stored after each of the iterations, wherein an identifier of the unlabeled images having the highest confidence score after each iteration is output.

30. The method of claim 27, further comprising receiving a data point label prior probability for the labeled and unlabeled image, wherein for each iteration of the calculations the data point label prior probability is adjusted according to an estimate of a data point class membership probability.

31. The method of claim 27, further comprising receiving a third unlabeled image of a face, comparing the third unlabeled image to at least some of the images having the highest confidence scores, and outputting an identifier of the third unlabeled image if a confidence that the face in the third unlabeled image is the same as the face in the seed image.