Computer-implemented system and method for visual document classification

• US 9,165,062 B2
• Filed: 01/17/2014
• Issued: 10/20/2015
• Est. Priority Date: 07/28/2009
• Status: Active Grant

• Current Assignee: Nuix North America Inc. (Nuix Ltd.)
• Original Assignee: FTI Consulting Incorporated
• Inventors: Knight, William C.
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US14/158,743
• Publication Number:
  US 20140136539A1
• Time in Patent Office: 641 Days
• Field of Search: 707/737, 707/E17.047, 707/E17.091
• US Class Current: 1/1
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A computer-implemented system and method for visual document classification are provided. One or more uncoded documents, each associated with a visual representation, are obtained. Reference documents, each associated with a classification code and a visual representation of that classification code, are obtained. At least one of the uncoded documents is compared to the reference documents and the reference documents similar to the uncoded document are identified based on the comparison. A suggestion for assigning one of the classification codes to the uncoded document based on the classification codes of the similar reference documents is provided, including displaying the visual representation of the suggested classification code placed on a portion of the visual representation associated with the at least one uncoded document. An acceptance of the suggested classification code is received and a size of the displayed visual representation of the accepted classification code is increased.

Claims

1. A computer-implemented system for visual document classification, comprising:

one or more uncoded documents, each associated with a visual representation;

one or more reference documents, each associated with a classification code and a visual representation of that classification code;

a processor to execute code, comprising;

a comparison module to compare at least one of the uncoded documents to one or more of the reference documents and to identify the reference documents similar to the at least one uncoded document based on the comparison;

a suggestion module to provide a suggestion for assigning one of the classification codes to the at least uncoded document based on the classification codes of the similar reference documents, comprising displaying the visual representation of the suggested classification code placed on a portion of the visual representation associated with the at least one uncoded document; and

an acceptance module to receive an acceptance of the suggested classification code and to increase a size of the displayed visual representation of the accepted classification code.

2. The computer-implemented system according to claim 1, further comprising:

a confidence module to determine a confidence level for the suggested classification code; and

a threshold module to compare the confidence level to a threshold,

wherein the visual representation of the suggested classification code is displayed when the confidence level exceeds the threshold.

3. The computer-implemented system according to claim 1, further comprising:

a difference module to receive from a user an assignment to the at least one uncoded document of one of the classification codes different from the suggested classification code; and

a display module to display the visual representation of the assigned classification code on a different portion of the visual representation associated with the at least one document.

4. The computer-implemented system according to claim 1, further comprising:

a confidence module to determine a confidence level for the suggested classification code;

a difference module to receive from a user an assignment to the at least one uncoded document of one of the classification codes different from the suggested classification code and to determine a confidence level for the assigned classification code;

a calculation module to calculate a difference between the confidence level for the suggested classification code and the confidence level for the assigned classification code and to compare the difference to a threshold; and

a discordance module to display a discordance between the assigned classification code and the suggested classification code when the difference exceeds the threshold.

5. The computer-implemented system according to claim 1, further comprising:

a clustering module to obtain one or more clusters, each comprising one or more of the uncoded documents; and

a cluster representation module to display the clusters.

6. The computer-implemented system according to claim 5, further comprising:

a similarity module to determine a similarity between the clusters;

an organization module to organize the clusters along one or more spines, each spine comprising a vector, based on the similarity; and

a spine display module to display the spines with the displayed clusters.

7. The computer-implemented system according to claim 5, further comprising:

a neighborhood module to compare a center of one of the displayed clusters to one or more of the reference documents and to determine a neighborhood of the reference documents for that displayed cluster based on the comparison; and

a cluster classification module to suggest one of the classification codes for assignment to the displayed cluster based on the classification codes of the reference documents in the neighborhood.

8. The computer-implemented system according to claim 7, further comprising:

a cluster suggestion module to display the visual representation of the classification code suggested for the displayed cluster on at least a portion of the displayed cluster.

9. The computer-implemented system according to claim 5, further comprising:

a compass module to provide a compass within which one or more of the clusters are displayed;

an emphasizing module to visually emphasize those of the clusters displayed within the compass; and

a deemphasizing module to visually deemphasize those of the clusters appearing outside of the compass.

10. The computer-implemented system according to claim 1, wherein the visual representation of the suggested classification code comprises at least one of a color, a pattern, a shape, and a symbol.

11. A computer-implemented method for visual document classification, comprising:

obtaining one or more uncoded documents, each associated with a visual representation;

obtaining one or more reference documents, each associated with a classification code and a visual representation of that classification code;

comparing at least one of the uncoded documents to one or more of the reference documents and identifying the reference documents similar to the at least one uncoded document based on the comparison;

providing a suggestion for assigning one of the classification codes to the at least uncoded document based on the classification codes of the similar reference documents, comprising displaying the visual representation of the suggested classification code placed on a portion of the visual representation associated with the at least one uncoded document; and

receiving an acceptance of the suggested classification code and increasing a size of the displayed visual representation of the accepted classification code.

12. The computer-implemented method according to claim 11, further comprising:

determining a confidence level for the suggested classification code; and

comparing the confidence level to a threshold,

wherein the visual representation of the suggested classification code is displayed when the confidence level exceeds the threshold.

13. The computer-implemented method according to claim 11, further comprising:

receiving from a user an assignment to the at least one uncoded document of one of the classification codes different from the suggested classification code; and

displaying the visual representation of the assigned classification code on a different portion of the visual representation associated with the at least one document.

14. The computer-implemented method according to claim 11, further comprising:

determining a confidence level for the suggested classification code; and

receiving from a user an assignment to the at least one uncoded document of one of the classification codes different from the suggested classification code;

determining a confidence level for the assigned classification code;

calculating a difference between the confidence level for the suggested classification code and the confidence level for the assigned classification code and comparing the difference to a threshold; and

displaying a discordance between the assigned classification code and the suggested classification code when the difference exceeds the threshold.

15. The computer-implemented method according to claim 11, further comprising:

obtaining one or more clusters, each comprising one or more of the uncoded documents; and

displaying the clusters.

16. The computer-implemented method according to claim 15, further comprising:

determining a similarity between the clusters;

organizing the clusters along one or more spines, each spine comprising a vector, based on the similarity; and

displaying the spines with the displayed clusters.

17. The computer-implemented method according to claim 15, further comprising:

comparing a center of one of the displayed clusters to one or more of the reference documents and determining a neighborhood of the reference documents for that displayed cluster based on the comparison; and

suggesting one of the classification codes for assigning to the displayed cluster based on the classification codes of the reference documents in the neighborhood.

18. The computer-implemented method according to claim 17, further comprising:

displaying the visual representation of the classification code suggested for the displayed cluster on at least a portion of the displayed cluster.

19. The computer-implemented method according to claim 15, further comprising:

providing a compass within which one or more of the clusters are displayed;

visually emphasizing those of the clusters displayed within the compass; and

visually deemphasizing those of the clusters appearing outside of the compass.

20. The computer-implemented method according to claim 11, wherein the visual representation of the suggested classification code comprises at least one of a color, a pattern, a shape, and a symbol.