System and method for providing a classification suggestion for electronically stored information

• US 8,635,223 B2
• Filed: 07/09/2010
• Issued: 01/21/2014
• Est. Priority Date: 07/28/2009
• Status: Active Grant

• Current Assignee: Nuix North America Inc. (Nuix Ltd.)
• Original Assignee: FTI Consulting Incorporated
• Inventors: Knight, William C.
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US12/833,769
• Publication Number:
  US 20110029525A1
• Time in Patent Office: 1,292 Days
• Field of Search: 707/737, 707/E17.046, 707/E17.047, 707/E17.089, 707/999.006, 707/999.107
• US Class Current: 707/737
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and method for providing a classification suggestion for electronically stored information is provided. A corpus of electronically stored information including reference electronically stored information items each associated with a classification and uncoded electronically stored information items are maintained. A cluster of uncoded electronically stored information items and reference electronically stored information items is provided. A neighborhood of reference electronically stored information items in the cluster is determined for at least one of the uncoded electronically stored information items. A classification of the neighborhood is determined using a classifier. The classification of the neighborhood is suggested as a classification for the at least one uncoded electronically stored information item.

Claims

1. A system for providing a classification suggestion for electronically stored information, comprising:

a database to store a corpus of electronically stored information (ESI) comprising reference ESI items each associated with a classification and uncoded ESI items;

a clustering engine to provide a cluster of uncoded ESI items and reference ESI items;

a neighborhood module to determine a neighborhood of reference ESI items in the cluster for at least one of the uncoded ESI items;

a classification module to determine a classification of the neighborhood using a classifier;

a suggestion module to suggest the classification of the neighborhood as a suggested classification code for the at least one uncoded ESI item;

a difference module to assign a further classification code to the at least one uncoded ESI item based on instructions from a user and to identify a difference between the assigned classification code and the suggested classification code;

a display module to display the difference between the assigned classification code and the suggested classification code; and

a processor to execute the modules.

2. The system according to claim 1, further comprising at least one of:

a marking module to mark the at least one uncoded ESI item based on the suggested classification with a visual indicator; and

an addition module to add the at least one uncoded ESI item to the corpus of ESI as a coded ESI item.

3. The system according to claim 1, further comprising a confidence module to provide a confidence level of the suggested classification code.

4. The system according to claim 3, further comprising a display to display the confidence level only when above a confidence level threshold.

5. The system according to claim 1, further comprising:

a distance module to determine a distance metric based on the similarity of each reference ESI item in the neighborhood to the at least one uncoded ESI item; and

an assign module to assign the classification of the reference ESI item in the neighborhood with the closest distance metric as the classification of the neighborhood.

6. The system according to claim 1, further comprising:

a distance module to determine a distance metric based on the similarity of each reference ESI item in the neighborhood to the at least one uncoded ESI item;

a calculation module to sum the distance metrics of the reference ESI items associated with the same classification and to average the sums of the distance metrics in each classification; and

an assign module to assign the classification of the reference ESI items in the neighborhood with the closest average distance metric as the classification of the neighborhood.

7. The system according to claim 1, further comprising:

a vote module to calculate a vote for each reference ESI item in the neighborhood; and

an assign module to assign the classification of the reference ESI items in the neighborhood with the highest calculated vote total as the classification of the neighborhood.

8. The system according to claim 1, further comprising:

a vote module to calculate a vote for each reference ESI item in the neighborhood;

a distance module to determine a distance metric based on the similarity of each reference ESI item in the neighborhood to the at least one uncoded ESI item;

a weight module to differentially weigh the votes based on the distance metric; and

an assign module to assign the classification of the reference ESI items in the neighborhood with the highest differentially weighted vote total as the classification of the neighborhood.

9. A method for providing a classification suggestion for electronically stored information, comprising the steps of:

maintaining a corpus of electronically stored information (ESI) comprising reference ESI items each associated with a classification and uncoded ESI items;

providing a cluster of uncoded ESI items and reference ESI items;

determining a neighborhood of reference ESI items in the cluster for at least one of the uncoded ESI items;

determining a classification of the neighborhood using a classifier;

suggesting the classification of the neighborhood as a suggested classification code for the at least one uncoded ESI item;

assigning a further classification code to the at least one uncoded ESI item based on instructions from a user;

identifying a difference between the assigned classification code and the suggested classification code; and

displaying the difference between the assigned classification code and the suggested classification code,

wherein the steps are performed by a suitably programmed computer.

10. The method according to claim 9, further comprising at least one of:

marking the at least one uncoded ESI item based on the suggested classification with a visual indicator; and

adding the at least one uncoded ESI item to the corpus of ESI as a coded ESI item.

11. The method according to claim 9, further comprising providing a confidence level of the suggested classification code.

12. The method according to claim 11, further comprising:

displaying the confidence level only when above a confidence level threshold.

13. The method according to claim 9, further comprising:

determining a distance metric based on the similarity of each reference ESI item in the neighborhood to the at least one uncoded ESI item; and

assigning the classification of the reference ESI item in the neighborhood with the closest distance metric as the classification of the neighborhood.

14. The method according to claim 9, further comprising:

determining a distance metric based on the similarity of each reference ESI item in the neighborhood to the at least one uncoded ESI item;

summing the distance metrics of the reference ESI items associated with the same classification;

averaging the sums of the distance metrics in each classification; and

assigning the classification of the reference ESI items in the neighborhood with the closest average distance metric as the classification of the neighborhood.

15. The method according to claim 9, further comprising:

calculating a vote for each reference ESI item in the neighborhood; and

assigning the classification of the reference ESI items in the neighborhood with the highest calculated vote total as the classification of the neighborhood.

16. The method according to claim 9, further comprising:

calculating a vote for each reference ESI item in the neighborhood;

determining a distance metric based on the similarity of each reference ESI item in the neighborhood to the at least one uncoded ESI item;

differentially weighing the votes based on the distance metric; and

assigning the classification of the reference ESI items in the neighborhood with the highest differentially weighted vote total as the classification of the neighborhood.

17. A system for providing a classification suggestion for a document, comprising:

a database to store a corpus of documents comprising reference documents each associated with a classification and uncoded documents;

a clustering engine to generate a cluster of uncoded documents;

a neighborhood module to determine a neighborhood of reference documents for at least one of the uncoded documents in the cluster;

a classification module to determine a classification of the neighborhood using a classifier; and

a suggestion module to suggest the classification of the neighborhood as a suggested classification code for the at least one uncoded document;

a difference module to assign a further classification code to the at least one uncoded document item based on instructions from a user and to identify a difference between the assigned classification code and the suggested classification code;

a display module to display the difference between the assigned classification code and the suggested classification code; and

a processor to execute the modules.

18. A system according to claim 17, further comprising:

a mark module to mark the at least one uncoded document with a different assigned classification code than the suggested classification code with a visual indicator.

19. The system according to claim 17, the display module further comprising:

a threshold module to display the difference only when above a discordance threshold.

20. The method according to claim 17, wherein the neighborhood is determined based on one of inclusion, injection, and nearest neighbor.

21. The system according to claim 17, wherein the classifier is one of minimum distance, minimum average distance, maximum counts, and distance weighted maximum count.

22. A method for providing a classification suggestion for a document, comprising the steps of:

maintaining a corpus of documents comprising reference documents each associated with a classification and uncoded documents;

generating a cluster of uncoded documents;

determining a neighborhood of reference documents for at least one of the uncoded documents in the cluster;

determining a classification of the neighborhood using a classifier; and

suggesting the classification of the neighborhood as a suggested classification code for the at least one uncoded document;

assigning a further classification code to the at least one uncoded document based on instructions from a user;

identifying a difference between the assigned classification code and the suggested classification code; and

displaying the difference between the assigned classification code and the suggested classification code,

wherein the steps are performed by a suitably programmed computer.

23. The method according to claim 22, further comprising:

marking the at least one uncoded ESI item with a different assigned classification code than the suggested classification code with a visual indicator.

24. The method according to claim 22, further comprising:

displaying the difference only when above a discordance threshold.

25. The method according to claim 22, wherein the neighborhood is determined based on one of inclusion, injection, and nearest neighbor.

26. The method according to claim 22, wherein the classifier is one of minimum distance, minimum average distance, maximum counts, and distance weighted maximum count.