Systems and methods for predictive coding

• US 7,933,859 B1
• Filed: 05/25/2010
• Issued: 04/26/2011
• Est. Priority Date: 05/25/2010
• Status: Active Grant

• Current Assignee: Open Text Holdings, Inc. (Open Text Corporation)
• Original Assignee: Recommind Incorporated (Open Text Corporation)
• Inventors: Vranas, Steve, Puzicha, Jan
• Primary Examiner(s): Fernandez Rivas; Omar F
• Application Number:
  US12/787,354
• Time in Patent Office: 336 Days
• Field of Search: None
• US Class Current: 706/62
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 5/048 Fuzzy inferencing
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Systems and methods for analyzing documents are provided herein. A plurality of documents and user input are received via a computing device. The user input includes hard coding of a subset of the plurality of documents, based on an identified subject or category. Instructions stored in memory are executed by a processor to generate an initial control set, analyze the initial control set to determine at least one seed set parameter, automatically code a first portion of the plurality of documents based on the initial control set and the seed set parameter associated with the identified subject or category, analyze the first portion of the plurality of documents by applying an adaptive identification cycle, and retrieve a second portion of the plurality of documents based on a result of the application of the adaptive identification cycle test on the first portion of the plurality of documents.

Claims

1. A method for analyzing a plurality of documents, comprising:

receiving the plurality of documents via a computing device;

receiving user input from the computing device, the user input including hard coding of a subset of the plurality of documents, the hard coding based on an identified subject or category;

executing instructions stored in memory, wherein execution of the instructions by a processor;

generates an initial control set based on the subset of the plurality of documents and the received user input on the subset,

analyzes the initial control set to determine at least one seed set parameter associated with the identified subject or category,

automatically codes a first portion of the plurality of documents, based on the initial control set and the at least one seed set parameter associated with the identified subject or category,

analyzes the first portion of the plurality of documents by applying an adaptive identification cycle, the adaptive identification cycle being based on the initial control set, user validation of the automated coding of the first portion of the plurality of documents and confidence threshold validation,

retrieves a second portion of the plurality of documents based on a result of the application of the adaptive identification cycle on the first portion of the plurality of documents;

adds further documents to the plurality of documents on a rolling load basis, and

conducts a random sampling of initial control set documents both on a static basis and the rolling load basis;

receiving user input via the computing device, the user input comprising inspection, analysis and hard coding of the randomly sampled initial control set documents; and

executing instructions stored in memory, wherein execution of the instructions by the processor automatically codes documents based on the received user input regarding the randomly sampled initial control set documents.

2. The method of claim 1, further comprising receiving user input from the computing device, the user input comprising a designation corresponding to key documents of the initial control set.

3. The method of claim 1, further comprising:

executing instructions stored in memory, wherein execution of the instructions by the processor;

automatically codes the second portion of the plurality of documents resulting from an application of user analysis and the adaptive identification cycle, and

adds the coded second portion of the plurality of documents to the initial control set.

4. The method of claim 1, further comprising transmitting to a display the first portion of the plurality of documents, the display coupled to the computing device.

5. The method of claim 1, further comprising:

receiving user input via the computing device, the user input corresponding to a confidence level; and

executing instructions stored in memory, wherein execution of the instructions by the processor;

calculates a statistic regarding machine-only accuracy rate, and

compares a statistic regarding machine coding accuracy rate against user input based on a defined confidence interval.

6. The method of claim 1, wherein the step of automatically coding a first portion of the plurality of documents further comprises executing instructions stored in memory, wherein execution of the instructions by the processor automatically codes based on probabilistic latent semantic analysis and support vector machine analysis of the first portion of the plurality of documents.