System and method for displaying relationships between concepts to provide classification suggestions via nearest neighbor

• US 8,645,378 B2
• Filed: 07/27/2010
• Issued: 02/04/2014
• Est. Priority Date: 07/28/2009
• Status: Active Grant

• Current Assignee: Nuix North America Inc. (Nuix Ltd.)
• Original Assignee: FTI Consulting Incorporated
• Inventors: Knight, William C., Nussbaum, Nicholas I., Conwell, John W.
• Primary Examiner(s): Chempakaseril, Ann
• Application Number:
  US12/844,813
• Publication Number:
  US 20110029532A1
• Time in Patent Office: 1,288 Days
• Field of Search: 707/737, 707/748, 707/73
• US Class Current: 707/737
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and method for displaying relationships between concepts to provide classification suggestions via nearest neighbor is provided. Reference concepts previously classified and a set of uncoded concepts are provided. At least one uncoded concept is compared with the reference concepts. One or more of the reference concepts that are similar to the at least one uncoded concept are identified. Relationships between the at least one uncoded concept and the similar reference concept are depicted on a display for classifying the at least one uncoded concept.

Claims

1. A method for displaying relationships between concepts to provide classification suggestions via nearest neighbor, comprising the steps of:

providing reference concepts each associated with a classification code and a set of uncoded concepts, wherein each of the reference concepts and the uncoded concepts comprises one or more nouns extracted from a plurality of documents;

associating each reference concept with a symbol representing the classification code;

comparing at least one uncoded concept with the reference concepts and identifying one or more of the reference concepts that are similar to the at least one uncoded concept;

displaying relationships between the at least one uncoded concept and the similar reference concepts for classifying the at least one uncoded concept based on the classification symbols of the similar reference concepts comprising at least one of;

displaying one or more of a presence and absence of the similar reference concepts with different classification codes; and

displaying a quantity of the similar reference concepts for each of the different classification codes;

providing at least one visual classification suggestion based on the displayed relationships; and

receiving a classification code for the at least one uncoded concept from a human reviewer based on the at least one suggestion,

wherein the steps are performed on a suitably programmed computer.

2. A method according to claim 1, further comprising:

classifying the at least one uncoded concept by assigning a classification code based on the relationships between the at least one uncoded concept and the similar reference concepts.

3. A method according to claim 2, further comprising:

adding the classified at least one uncoded concept to the reference concepts.

4. A method according to claim 2, further comprising:

providing a confidence level for the classification code of the at least one uncoded concept.

5. A method according to claim 2, further comprising:

identifying the documents associated with the at least one classified concept; and

assigning the classification code for the at least one classified concept to one or more of the associated documents.

6. A method according to claim 5, wherein the documents are identified using a matrix comprising a mapping of concepts and related documents.

7. A method according to claim 1, further comprising:

generating the reference concepts from a set of concepts, comprising at least one of;

identifying the concepts that are dissimilar from each other concept in the set of concepts and assigning the classification code to each of the dissimilar concepts, as the reference concepts; and

grouping the set of concepts into clusters, selecting one or more of the concepts in at least one cluster, and assigning the classification code to each of the selected concepts, as the reference concepts.

8. A method according to claim 1, further comprising:

determining the similar reference concepts, comprising;

forming a score vector for each uncoded concept and each reference concept; and

calculating a similarity metric by comparing the score vectors for the at least one uncoded concept and each of the reference concepts; and

selecting the reference concepts with the highest similarity metrics as the similar reference concepts.

9. A method according to claim 1, further comprising:

determining the similar reference concepts, comprising;

determining a measure of similarity between the at least one uncoded concept and each of the reference concepts based on the comparison;

applying a threshold to the measures of similarity; and

selecting those reference concepts that satisfy the threshold as the similar reference concepts.

10. A method according to claim 1, further comprising:

clustering the uncoded concepts and displaying the clusters; and

displaying the similar reference concepts in a list adjacent to the clusters.

11. A system for displaying relationships between concepts to provide classification suggestions via nearest neighbor, comprising:

a database to maintain reference concepts each associated with a classification code and a set of uncoded concepts, wherein each of the reference concepts and the uncoded concepts comprises one or more nouns extracted from a plurality of documents;

an association module to associate each reference concept with a symbol representing the classification code;

a similarity module to compare at least one uncoded concept with the reference concepts and to identify one or more of the reference concepts that are similar to the at least one uncoded concept;

a display to display relationships between the at least one uncoded concept and the similar reference concepts for classifying the at least one uncoded concept based on the classification symbols of the similar reference concepts comprising at least one of;

a presence module to display one or more of a presence and absence of the similar reference concepts with different classification codes; and

a quantity module to display a quantity of the similar reference concepts for each of the different classification codes;

a suggestion module to provide at least one visual classification suggestion based on the displayed relationships;

a classification module to receive a classification code for the at least one uncoded concept from a human reviewer based on the at least one suggestion; and

a computer processor to execute the modules.

12. A system according to claim 11, further comprising:

a classification module to classify the at least one uncoded concept by assigning a classification code based on the relationships between the at least one uncoded concept and the similar reference concepts.

13. A system according to claim 12, further comprising:

a reference module to add the classified at least one uncoded concept to the reference concepts.

14. A system according to claim 12, wherein the classification module provides a confidence level for the classification code of the at least one uncoded concept.

15. A system according to claim 12, further comprising:

a document classification module to identify the documents associated with the at least one classified concept and to assign the classification code for the at least one classified concept to one or more of the associated documents.

16. A system according to claim 15, wherein the documents are identified using a matrix comprising a mapping of concepts and related documents.

17. A system according to claim 11, further comprising:

a reference set module to generate the reference concepts from a set of concepts, comprising at least one of;

a comparison module to identify the concepts that are dissimilar from each other concept in the concept set and to assign the classification code to each of the dissimilar concepts, as the reference concepts; and

a reference clustering module to group the set of concepts into one or more clusters, to select one or more of the concepts in at least one cluster, and to assign the classification code to each of the selected concepts, as the reference concepts.

18. A system according to claim 11, further comprising:

a concept similarity module to determine the similar reference concepts, comprising;

a vector module to form a score vector for each uncoded concept and each reference concept; and

a similarity measurement module to calculate a similarity metric by comparing the score vectors for the at least one uncoded concept and each of the reference concepts and to select the reference concepts with the highest similarity metrics as the similar reference concepts.

19. A system according to claim 11, further comprising:

a concept similarity module to determine the similar reference concepts, comprising;

a similarity measurement module to determine a measure of similarity between the at least one uncoded concept and each of the reference concepts based on the comparison; and

a threshold module to apply a threshold to the measures of similarity and to select those reference concepts that satisfy the threshold as the similar reference concepts.

20. A system according to claim 11, further comprising:

a clustering module to cluster the uncoded concepts; and

the display to present the clusters and the similar reference concepts in a list adjacent to the clusters.