System and method for displaying relationships between concepts to provide classification suggestions via injection

• US 9,477,751 B2
• Filed: 07/27/2010
• Issued: 10/25/2016
• Est. Priority Date: 07/28/2009
• Status: Expired due to Fees

• Current Assignee: Nuix North America Inc. (Nuix Ltd.)
• Original Assignee: FTI Consulting Incorporated
• Inventors: Knight, William C., Nussbaum, Nicholas I., Conwell, John W.
• Primary Examiner(s): Shanmugasundaram, Kannan
• Application Number:
  US12/844,792
• Publication Number:
  US 20110029530A1
• Time in Patent Office: 2,282 Days
• Field of Search: 707/737, 707999001-999005
• US Class Current: 1/1
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and method for displaying relationships between concepts to provide classification suggestions via injection is provided. A reference set of concepts each associated with a classification code is designated. Clusters of uncoded concepts are designated. One or more of the uncoded concepts from at least one cluster are compared to the reference set. At least one of the concepts in the reference set that is similar to the one or more uncoded concepts is identified. The similar concepts are injected into the at least one cluster. Relationships between the uncoded concepts and the similar concepts in the at least one cluster are visually depicted as suggestions for classifying the uncoded concepts.

Claims

1. A method for displaying relationships between concepts to provide classification suggestions via injection, comprising the steps of:

designating reference concepts each associated with one of a plurality of classification codes, wherein each concept comprises nouns and noun phrases with common semantic meaning that are extracted from a set of documents;

designating clusters of uncoded concepts;

comparing one or more of the uncoded concepts from at least one cluster of the uncoded concepts to the coded reference concepts;

identifying at least one of the coded reference concepts that is similar to the one or more uncoded concepts in the at least one cluster;

injecting the similar coded reference concepts into the at least one cluster of the uncoded concepts;

displaying each of the uncoded concepts and the coded reference concepts in the at least one cluster via a representation;

providing a classification suggestion for one or more of the uncoded concepts based on a presence of one or more different classification codes of the coded reference concepts and an absence of one or more of the different classification codes of the coded reference concepts;

receiving from a human reviewer a classification code for at least one of the uncoded concepts associated with the classification suggestion, wherein the received classification code is one of the same as or different than the classification suggestion,

wherein the steps are performed by a suitably programmed computer.

2. A method according to claim 1, further comprising:

determining the similar coded reference concepts, comprising;

calculating a similarity between each of the one or more uncoded concepts and each of the coded reference concepts;

applying a predetermined threshold to the calculated similarities; and

selecting the at least one coded reference concept as the similar coded reference concept when the associated similarity satisfies the similarity threshold.

3. A method according to claim 1, wherein the comparison between the one or more of the uncoded concepts and the coded reference concepts comprise at least one of:

determining a center of the at least one cluster based on the one or more uncoded concepts and comparing the cluster center with the coded reference concepts;

selecting a sample comprising the one or more uncoded concepts and comparing the sample with the coded reference concepts, wherein the sample can be selected via at least one of manually and automatically; and

comparing the cluster center and the sample for the at least one cluster with the coded reference concepts.

4. A method according to claim 1, further comprising:

determining a classification code for at least one of the uncoded concepts in the at least one cluster, comprising;

comparing the at least one uncoded concept to a neighborhood of coded reference concepts in the at least one cluster, wherein the neighborhood comprises a predetermined number of the coded reference concepts that have a closest distance to the uncoded concept; and

selecting the classification code based on the comparison of the at least one uncoded concept and the coded reference concept neighborhood.

5. A method according to claim 4, further comprising:

automatically classifying the at least one uncoded concept by assigning the classification code to that uncoded concept.

6. A method according to claim 5, further comprising:

identifying the documents associated with the classified uncoded concept; and

classifying the associated documents by assigning a classification code to each of the documents.

7. A method according to claim 6, wherein the documents are identified using a matrix comprising a mapping of concepts and related documents.

8. A method according to claim 4, further comprising:

providing the classification code as the suggestion for classifying the at least one uncoded concept in the at least one cluster.

9. A method according to claim 8, further comprising:

determining a confidence level for the suggested classification code; and

providing the confidence level with the suggested classification code.

10. A method according to claim 9, wherein the classification code is provided when the confidence level exceeds a predetermined threshold.

11. A system for displaying relationships between concepts to provide classification suggestions via injection, comprising:

reference concepts each associated with one of a plurality of classification codes, wherein each concept comprises nouns and noun phrases with common semantic meaning that are extracted from a set of documents; and

a processor configured to designate clusters of uncoded concepts, to compare one or more of the uncoded concepts from at least one cluster of the uncoded concepts to the coded reference concepts in the set, to identify at least one of the coded reference concepts that is similar to the one or more uncoded concepts in the at least one cluster, to inject the similar coded reference concepts into the at least one cluster of the uncoded concepts, to provide a classification suggestion for one or more of the uncoded concepts based on a presence of one or more different classification codes of the coded reference concepts and an absence of one or more of the different classification codes of the coded reference concept, to display each of the uncoded concepts and the coded reference concepts in the at least one cluster via a representation, to display each suggestion within the representation for that uncoded concept, and to receive from a human reviewer a classification code for at least one of the uncoded concepts associated with the classification suggestion, wherein the received classification code is one of the same as or different than the classification suggestion.

12. A system according to claim 11, wherein the processor is further configured to determine the similar coded reference concepts by calculating a similarity between each of the one or more uncoded concepts and each of the coded reference concepts, by applying a predetermined threshold to the calculated similarities, and by selecting the at least one coded reference concept as the similar coded reference concept when the associated similarity satisfies the similarity threshold.

13. A system according to claim 11, wherein the processor is further configured to determine a center of the at least one cluster based on the one or more uncoded concepts, to compare the cluster center with the coded reference concepts, to select a sample comprising the one or more uncoded concepts, to compare the sample with the coded reference concepts, wherein the sample can be selected via at least one of manually and automatically, and to compare the cluster center and the sample for the at least one cluster with the coded reference concepts.

14. A system according to claim 11, wherein the processor is further configured to determine a classification code for at least one of the uncoded concepts in the at least one cluster, by comparing the at least one uncoded concept to a neighborhood of coded reference concepts in the at least one cluster, wherein the neighborhood comprises a predetermined number of the coded reference concepts that have a closest distance to the uncoded concept and by selecting the classification code based on the comparison of the at least one uncoded concept and the coded reference concept neighborhood.

15. A system according to claim 14, wherein the processor automatically classifies the at least one uncoded concept by assigning the classification code to that uncoded concept.

16. A system according to claim 15, wherein the processor identifies the documents associated with the classified uncoded concept and classifies the associated documents by assigning a classification code to each of the documents.

17. A system according to claim 16, wherein the documents are identified using a matrix comprising a mapping of concepts and related documents.

18. A system according to claim 14, wherein the processor provides the classification code as the suggestion for classifying the at least one uncoded concept in the at least one cluster.

19. A system according to claim 18, wherein the processor determines a confidence level for the suggested classification code and provides the confidence level with the suggested classification code.

20. A system according to claim 19, wherein the classification code is provided when the confidence level exceeds a predetermined threshold.