System And Method For Displaying Relationships Between Electronically Stored Information To Provide Classification Suggestions Via Nearest Neighbor

• US 20110029527A1
• Filed: 07/09/2010
• Published: 02/03/2011
• Est. Priority Date: 07/28/2009
• Status: Active Grant

• Current Assignee: Nuix North America Inc. (Nuix Ltd.)
• Original Assignee: FTI Consulting Incorporated
• Inventors: Nussbaum, Nicholas I., Knight, William C.
• Granted Patent: US 8,572,084 B2
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and for providing reference documents as a suggestion for classifying uncoded documents is provided. Reference electronically stored information items and a set of uncoded electronically stored information items are designated. Each of the reference information items are previously classified. At least one uncoded electronically stored information item is compared with the reference electronically stored information items. One or more of the reference electronically stored information items similar to the at least one uncoded electronically stored information items are identified. Relationships are depicted between the at least one uncoded electronically stored information item and the similar reference electronically stored information items for classifying the at least one uncoded electronically stored information item.

Claims

1. A system for providing reference documents as a suggestion for classifying electronically stored information using nearest neighbor, comprising:

a clustering module to provide reference electronically stored information items previously classified and a set of uncoded electronically stored information items;

a similarity module to compare at least one uncoded electronically stored information item with the reference electronically stored information items and to identify one or more of the reference electronically stored information items that are similar to the at least one uncoded electronically stored information item; and

a display to depict relationships between the at least one uncoded electronically stored information item and the similar reference electronically stored information items for classifying the at least one uncoded electronically stored information item.

2. A system according to claim 1, further comprising:

a reference set module to generate the set of reference electronically stored information items, comprising at least one of;

a comparison module to obtain a set of electronically stored information items, to identify one or more electronically stored information items that are dissimilar from each other electronically stored information item, and to assign a classification code to each of the dissimilar electronically stored information items, as the reference electronically stored information items; and

a reference clustering module to group electronically stored information items for a document review project into one or more clusters, to select one or more of the electronically stored information items in at least one cluster, and to assign a classification code to each of the selected electronically stored information items, as the reference electronically stored information items.

3. A system according to claim 1, further comprising:

a score module to form a score vector for each uncoded electronically stored information item and each reference electronically stored information item; and

the similarity module to calculate the similarity metric by comparing the score vectors for the uncoded electronically stored information items and the reference electronically stored information items.

4. A system according to claim 1, wherein the similarity metric is calculated according to the following equation:

5. A system according to claim 1, further comprising:

a classification module to assign a classification code to one or more of the uncoded electronically stored information items.

6. A method for providing reference documents as a suggestion for classifying electronically stored information using nearest neighbor, comprising:

designating reference electronically stored information items, which are previously classified, and a set of uncoded electronically stored information items;

comparing at least one uncoded electronically stored information item with the reference electronically stored information items;

identifying one or more of the reference electronically stored information items that are similar to the at least one uncoded electronically stored information item; and

depicting relationships between the at least one uncoded electronically stored information item and the similar reference electronically stored information items for classifying the at least one uncoded electronically stored information item.

7. A method according to claim 6, further comprising:

generating the reference electronically stored information items from a set of electronically stored information items, comprising at least one of;

identifying the electronically stored information items that are dissimilar from each other electronically stored information item and assigning a classification code to each of the dissimilar electronically stored information items, as the reference electronically stored information items; and

grouping a set of electronically stored information items associated with a document review project into one or more clusters, selecting one or more of the electronically stored information items in at least one cluster, and assigning a classification code to each of the selected electronically stored information items, as the reference electronically stored information items.

8. A method according to claim 6, further comprising:

forming a score vector for each uncoded electronically stored information item and each reference electronically stored information item; and

calculating a similarity metric by comparing the score vectors for the uncoded electronically stored information items and the reference electronically stored information items in the reference set.

9. A method according to claim 8, wherein the similarity metric is calculated according to the following equation:

10. A method according to claim 6, further comprising:

assigning a classification code to one or more of the uncoded electronically stored information items based on the similar reference electronically stored information items.

11. A system for identifying reference documents for use in classifying uncoded documents, comprising:

a database to store a set of reference documents each associated with a classification code;

a clustering module to designate a set of clusters each comprising uncoded documents;

a similarity module to select at least one uncoded document, to compare the at least one uncoded document with each of the reference documents, and to identify one or more reference documents that satisfy a threshold of similarity with the at least one uncoded document; and

a display to display relationships between the at least one uncoded document and the similar reference documents based on the associated classification codes as suggestions for classifying the at least one uncoded document.

12. A system according to claim 11, further comprising:

a reference set module to generate the set of reference documents, comprising at least one of;

a reference similarity module to obtain a set of documents, to identify the documents that are dissimilar from each other document, and to assign a classification code to each of the dissimilar documents; and

a reference cluster module to generate clusters of documents for a document review project, to select one or more of the documents in at least one of the clusters, and to assign a classification code to each of the documents.

13. A system according to claim 11, wherein the one or more reference documents are selected from at least one of a predefined, customized, or arbitrary reference document set.

14. A system according to claim 11, wherein the similarity module comprises:

a score module to form a score vector for each uncoded document and each reference document; and

a vector similarity module to calculate a similarity metric between the score vectors for the uncoded documents and reference documents.

15. A system according to claim 11, wherein the similarity metric is calculated according to the following equation:

16. A method for identifying reference documents for use in classifying uncoded documents, comprising:

designating a set of reference documents each associated with a classification code;

designing a set of clusters each comprising uncoded documents;

selecting at least one uncoded document and comparing the at least one uncoded document with each of the reference documents;

identifying one or more reference documents that satisfy a threshold of similarity with the at least one uncoded document; and

displaying relationships between the at least one uncoded document and the similar reference documents based on the associated classification codes as suggestions for classifying the at least one uncoded document.

17. A method according to claim 16, further comprising:

generating the set of reference documents, comprising at least one of;

obtaining a set of documents, identifying the documents that are dissimilar from each other document, and assigning a classification code to each of the dissimilar documents; and

generating clusters of documents, selecting one or more of the documents in at least one of the clusters and assigning a classification code to each of the documents.

18. A method according to claim 16, wherein the one or more reference documents are selected from at least one of a predefined, customized, or arbitrary reference document set.

19. A method according to claim 16, further comprising:

forming a score vector for each uncoded document and cach reference document; and

calculating a similarity metric between the score vectors for the uncoded documents and reference documents.

20. A method according to claim 16, wherein the similarity metric is calculated according to the following equation: