System and method for displaying relationships between concepts to provide classification suggestions via inclusion

• US 8,700,627 B2
• Filed: 07/27/2010
• Issued: 04/15/2014
• Est. Priority Date: 07/28/2009
• Status: Active Grant

• Current Assignee: Nuix North America Inc. (Nuix Ltd.)
• Original Assignee: FTI Consulting Incorporated
• Inventors: Knight, William C., Nussbaum, Nicholas I., Conwell, John W.
• Primary Examiner(s): Chempakaseril, Ann
• Application Number:
  US12/844,810
• Publication Number:
  US 20110029531A1
• Time in Patent Office: 1,358 Days
• Field of Search: 707/737, 707/748
• US Class Current: 707/737
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and for displaying relationships between concepts to provide classification suggestions via inclusion is provided. A set of reference concepts each associated with a classification code is designated. One or more of the reference concepts are combined with a set of uncoded concepts. Clusters of the uncoded concepts and the one or more reference concepts are generated. Relationships between the uncoded concepts and the one or more reference concepts in at least one cluster are visually depicted as suggestions for classifying the uncoded concepts in that cluster.

Claims

1. A method for displaying relationships between concepts to provide classification suggestions via inclusion, comprising the steps of:

designating reference concepts each associated with a classification code and a visual representation of that classification code comprising at least one of a shape, a color, and a symbol, wherein each concept comprises nouns and noun phrases with common semantic meaning that are extracted from a set of documents;

extracting uncoded concepts from unclassified documents and associating each of the uncoded concepts with a visual representation different from the representations of the classification codes;

forming a grouped concept set by grouping a subset of the classified reference concepts with a set of uncoded concepts;

generating clusters, each comprising a portion of the uncoded concepts and the classified reference concepts of the grouped concept set, comprising;

determining a similarity between the concepts in the grouped concept set; and

putting the concepts whose similarity exceeds a threshold into one of the clusters;

visually depicting relationships between the uncoded concepts and one or more classified reference concepts in at least one of the clusters as suggestions for classifying the uncoded concepts in that cluster, comprising displaying the visual representation associated with each of the classified reference concepts in that cluster and the visual representation of each of the uncoded concepts in that cluster,

wherein the steps are performed by a suitably programmed computer.

2. A method according to claim 1, further comprising:

determining a classification code for at least one of the uncoded concepts in the at least one cluster, comprising;

comparing the at least one uncoded concept to a neighborhood of classified reference concepts in the at least one cluster, wherein the neighborhood comprises a predetermined number of the classified reference concepts that have a closest distance to the uncoded concept; and

selecting the classification code based on the comparison of the at least one uncoded concept and classified reference concept neighborhood.

3. A method according to claim 1, further comprising:

automatically classifying the at least one uncoded concept by assigning the classification code to that uncoded concept.

4. A method according to claim 3, further comprising:

identifying the documents associated with the classified uncoded concept; and

classifying the associated documents by assigning a classification code to each of the documents.

5. A method according to claim 4, wherein the documents are identified using a matrix comprising a mapping of concepts and related documents.

6. A method according to claim 2, further comprising:

providing the classification code as a suggestion for classifying the at least one uncoded concept in the at least one cluster.

7. A method according to claim 6, further comprising:

providing a confidence level for the classification code suggestion.

8. A method according to claim 7, wherein the classification code is provided when the confidence level exceeds a predetermined threshold.

9. A method according to claim 1, further comprising:

generating the set of classified reference concepts, comprising at least one of;

identifying dissimilar uncoded concepts for a document review project and assigning a classification code to each of the dissimilar uncoded concepts for inclusion in the reference concepts set; and

clustering uncoded concepts for a document review project, selecting one or more of the uncoded concepts in one or more of the clusters, and assigning a classification code to each of the selected uncoded concepts for inclusion in the reference concepts set.

10. A method according to claim 1, wherein the subset of classified reference concepts is determined via at least one of predefined, arbitrary, or customized selection.

11. A system for displaying relationships between concepts to provide classification suggestions via inclusion, comprising:

reference concepts each associated with a classification code and a visual representation of that classification code comprising at least one of a shape, a color, and a symbol, wherein each concept comprises nouns and noun phrases with common semantic meaning that are extracted from a set of documents;

an extraction module to extract uncoded concepts from unclassified documents and to associate each of the uncoded concepts with a visual representation different from the representations of the classification codes;

a clustering module to form a grouped concept set by grouping a subset of the coded reference concepts with a set of uncoded concepts and to generate clusters, each comprising a portion of the uncoded concepts and the classified reference concepts of the grouped concept set, comprising;

a similarity module to determine a similarity between the concepts in the grouped concept set; and

a threshold module putting the concepts whose similarity exceeds a threshold into one of the clusters;

a display to visually depict relationships between the uncoded concepts and one or more classified reference concepts in at least one of the clusters as suggestions for classifying the uncoded concepts in that cluster by displaying the visual representation of the classification code of each of the classified reference concepts in that cluster and the visual representation of each of the uncoded concepts in that cluster; and

a processor to execute the modules.

12. A system according to claim 11, further comprising:

a classification module to determine a classification code for at least one of the uncoded concepts in the at least one cluster, comprising;

a comparison module to compare the at least one uncoded concept to a neighborhood of classified reference concepts in the at least one cluster, wherein the neighborhood comprises a predetermined number of the classified reference concepts that have a closest distance to the uncoded concept; and

a selection module to select the classification code based on the comparison of the at least one uncoded concept and classified reference concept neighborhood.

13. A system according to claim 11, wherein the classification module automatically classifies the at least one uncoded concept by assigning the classification code to that uncoded concept.

14. A system according to claim 13, wherein the classification module identifies the documents associated with the classified uncoded concept and classifies the associated documents by assigning a classification code to each of the documents.

15. A system according to claim 14, wherein the documents are identified using a matrix comprising a mapping of concepts and related documents.

16. A system according to claim 12, wherein the classification module provides the classification code as a suggestion for classifying the at least one uncoded concept in the at least one cluster.

17. A system according to claim 16, wherein the classification module provides a confidence level with the classification code suggestion.

18. A system according to claim 17, wherein the classification code is provided when the confidence level exceeds a predetermined threshold.

19. A system according to claim 11, further comprising:

a reference module to generate the set of classified reference concepts, comprising at least one of;

a reference similarity module to identify dissimilar uncoded concepts for a document review project and to assign a classification code to each of the dissimilar uncoded concepts for inclusion in the reference concepts set; and

a reference clustering module to cluster uncoded concepts for a document review project, to select one or more of the concepts in at least one cluster, and to assign a classification code to each of the selected concepts for inclusion in the reference concepts set.

20. A system according to claim 11, wherein the subset of classified reference concepts is determined via at least one of predefined, arbitrary, or customized selection.