Predictive coding of documents in an electronic discovery system

• US 8,504,489 B2
• Filed: 03/24/2010
• Issued: 08/06/2013
• Est. Priority Date: 03/27/2009
• Status: Active Grant

• Current Assignee: Bank of America Corp.
• Original Assignee: Bank of America Corp.
• Inventors: Richards, Phillip L., Andersen, David M., Miller, Emerson D., Clark, Benjamin, Knox, Jeffrey V., Mayer, Michael J.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Sugent, James F
• Application Number:
  US12/730,780
• Publication Number:
  US 20100250474A1
• Time in Patent Office: 1,231 Days
• Field of Search: 706/12, 707/603, 707/706, 707/736, 707/737, 707/758, 707/777
• US Class Current: 706/12
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Embodiments of the invention relate to systems, methods, and computer program products for improved electronic discovery. More specifically, embodiments relate to computer program products for predictive and automated coding of identical or highly similar documents for the purpose of limiting the volume of documents requiring review and thereby increasing the overall efficiency of the document review process.

Claims

1. A method for predictive coding of previously collected electronic discovery documents, the method comprising:

receiving, at a computing device, a plurality of document coding inputs that each assign a review code to one of a plurality of first documents, which are associated with a case within an electronic discovery system and have been previously collected from one of a plurality of custodians associated with the case, wherein the review code indicates a level of relevancy or importance in relation to the case and at least one document coding input codes a first document as privileged;

in response to receiving each of the plurality of document coding inputs, determining, via a computing device processor, if one or more second documents, which are associated with the case, have been previously collected from the plurality of custodians and are pending review, are similar to or same as the first document;

in response to receiving the at least one document coding input that codes the first document as privileged, determining, via a computing device, if one or more third documents, which are associated with other cases in the electronic discovery system, have been collected from the plurality of custodians and are pending review are same as the first document;

in response to determining that the second documents are same as the first document, automatically assigning, via a computing device processor, a review code assigned to the first document to the one or more second documents that are the same as the first document;

in response to determining that the third documents are same as the first document, automatically assigning, via a computing device processor, the privilege code to the one or more third documents that are the same as the first document;

removing, via a computing device processor, the one or more second documents that are the same as the first document from a plurality of pending review documents based on the assignment of the review code, wherein the pending review documents are included in a document review assignment currently being reviewed by a reviewer; and

presenting, via a computing device display, the one or more second documents that are similar to the first document and a confidence indicator that indicates a level of similarity between the first document and a presented second document, wherein the reviewer makes a determination based on the confidence indicator as to whether the presented second document reaches a level of similarity to the first document to justify assigning the review code to the presented document.

2. The method of claim 1, wherein removing further comprises removing, via the computing device processor, the one or more second documents that are the same as the first document from the plurality of pending review documents, wherein removing occurs in near real-time to receiving the document coding input.

3. The method of claim 1, wherein determining if the one or more second documents are the same as the first document further comprises determining, via the computing device processor, if the one or more second documents that are the same as the first document have a same hash mark as the first document.

4. The method of claim 1, wherein determining further comprises providing, via computing device processor, probability analysis to determine the one or more second documents associated with the case are similar to or the same as the first document.

5. The method of claim 4, wherein providing, further comprises providing, via the computing device processor, at least one of clustering technique analysis, machine learning or Bayesian technique analysis to determine the one or more second documents associated with the case are similar to or same as the first document.

6. The method of claim 1, wherein determining further comprises determining, via the computing device, if the one or more second documents include a predetermined threshold of words or phrases included in the first document.

7. The method of claim 1, wherein determining further comprises determining, via the computing device, if the one or more second documents include a predetermined threshold of concepts included in the first document.

8. The method of claim 1, wherein receiving further comprises receiving, at the computing device, the document coding input that assigns the review code, wherein the review code includes a mark.

9. The method of claim 1, wherein receiving further comprises receiving, at the computing device, the document coding input that assigns the review code, wherein the review code includes a mark and at least one tag.

10. An apparatus for predictive coding of previously collected electronic discovery documents, the apparatus comprising:

a computing platform including at least one processor, a memory in communication with the processor and a display in communication with the processor;

a document coding application stored in the memory, executable by the processor and configured to receive a plurality of document coding inputs that each assign a review code to one of a plurality of first documents, which are associated with a case within an electronic discovery system and have been previously collected from one of a plurality of custodians associated with the case, wherein the review code indicates a level of relevancy or importance in relation to the case and at least one document coding input codes a first document as privileged; and

a predictive document coding application stored in the memory, executable by the processor and including,

a document similarity evaluator configured to (1) determine, in response to receiving each of the plurality of document coding inputs, if one or more second documents, which are associated with the case, have been previously collected from the plurality of custodians and are pending review, are similar to or same as the first document, and (2) determine, in response to receiving the at least one document coding input that codes the first document as privileged, determining, via a computing device, if one or more third documents, which are associated with other cases in the electronic discovery system, have been collected from the plurality of custodians and are pending review are same as the first document,

a code assignor configured to (1), in response to determining that the second documents are same as the first document, automatically assign a review code assigned to the first document to the one or more second documents that are the same as the first document and, (2) in response to determining that the third documents are same as the first document, automatically assign the privilege code to the one or more third documents that are the same as the first document,

a document remover configured to remove the one or more second documents that are the same as the first document from a plurality of pending review documents based on the assignment of the review code, wherein the pending review documents are included in a document review assignment currently being reviewed by a reviewer, and

a confidence indicator presenter configured to present, on the display, the one or more second documents that are similar to the first document and a confidence indicator that indicates a level of similarity between the first document and a presented second document, wherein the reviewer makes a determination based on the confidence indicator as to whether the presented second document reaches a level of similarity to the first document to justify assigning the review code to the presented document.

11. The apparatus of claim 10, wherein the document remover is further configured to remove the one or more second documents that are the same as the first document from the plurality of pending review documents, wherein removing occurs in near real-time to the document coding application receiving the document coding input.

12. The apparatus of claim 10, wherein the document similarity evaluator is further configured to determine that the one or more second documents that are the same as the first document have a same hash mark as the first document.

13. The apparatus of claim 10, wherein the document similarity evaluator is further configured to perform probability analysis to determine the one or more second documents associated with the case are at least similar to or same as the first document.

14. The apparatus of claim 13, wherein the document similarity evaluator is further configured to perform at least one of clustering technique analysis, machine learning or Bayesian technique analysis to determine the one or more second documents associated with the case are similar to or same as the first document.

15. The apparatus of claim 10, wherein the document similarity evaluator is further configured to determine if the one or more second documents include a predetermined threshold of words or phrases included in the first document.

16. The apparatus of claim 10, wherein the document similarity evaluator is further configured to determine if the one or more second documents include a predetermined threshold of concepts included in the first document.

17. The apparatus of claim 10, wherein the document coding application is further configured to receive the document coding input that assigns the review code, wherein the review code includes a mark.

18. The apparatus of claim 10, wherein the document coding application is further configured to receive the document coding input that assigns the review code, wherein the review code includes a mark and at least one tag.

19. A computer program product comprising:

a non-transitory computer-readable medium comprising;

a first set of codes for causing a computer to receive a plurality of document coding inputs that each assign a review code to one of a plurality of first documents, which are associated with a case within an electronic discovery system and have been previously collected from one of a plurality of custodians associated with the case, wherein the review code indicates a level of relevancy or importance in relation to the case and at least one document coding input codes a first document as privileged;

a second set of codes for causing a computer to (1), in response to receiving each of the plurality of document coding inputs, determine if one or more second documents, which are associated with the case, have been previously collected from the plurality of custodians and are pending review, are similar to or same as the first document and (2) in response to receiving the at least one document coding input that codes the first document as privileged determine if one or more third documents which are associated with other cases in the electronic discovery system, have been collected from the plurality of custodians and are pending review are same as the first document;

third set of codes for causing a computer to (1), in response to determining that the second documents are same as the first document, automatically assign a review code assigned to the first document to the one or more second documents that are the same as the first document and (2), in response to determining that the third documents are same as the first document, automatically assign the privilege code to the one or more third documents that are the same as the first document;

a fourth set of codes for causing a computer to remove the one or more second documents from a plurality of pending review documents based on the assignment of the review code, wherein the pending review documents are included in a document review assignment currently being reviewed by a reviewer; and

a fifth set of codes for causing a computer to present, on a computing device display, the one or more second documents that are similar to the first document and a confidence indicator that indicates a level of similarity between the first document and a presented second document, wherein the reviewer makes a determination based on the confidence indicator as to whether the presented second document reaches a level of similarity to the first document to justify assigning the review code to the presented document.

20. The computer program product of claim 19, wherein the fourth set of codes is further configured to cause the computer to remove, in near real-time to receiving the document coding input, the one or more second documents that are the same as the first document from the plurality of pending review documents, wherein removing occurs in near real-time to receiving the document coding input.

21. The computer program product of claim 19, wherein the second set of codes is further configured to cause the computer to determine if one or more second documents that are the same as the first document have a same hash mark as the first document.

22. The computer program product of claim 19, wherein the second set of codes is further configured to cause the computer to perform probability analysis to determine the one or more second documents associated with the case are similar to or the same as the first document.

23. The computer program product of claim 22, wherein the second set of codes is further configured to cause the computer to perform at least one of clustering technique analysis, machine learning or Bayesian technique analysis to determine that the one or more second documents associated with the case are at least similar to or same as the first document.

24. The computer program product of claim 19, wherein the second set of codes is further configured to cause the computer to determine if the one or more second documents include a predetermined threshold of words or phrases included in the first document.

25. The computer program product of claim 19, wherein the second set of codes is further configured to cause the computer to determine if the one or more second documents include a predetermined threshold of concepts included in the first document.

26. The computer program product of claim 25, wherein the first set of codes is further configured to cause the computer to receive the document coding input that assigns the review code, wherein the review code includes a mark.

27. The computer program product of claim 25, wherein the first set of codes is further configured to cause the computer to receive the document coding input that assigns the review code, wherein the review code includes a mark and at least one tag.