×

Systems and methods for conducting a highly autonomous technology-assisted review classification

  • US 10,229,117 B2
  • Filed: 06/17/2016
  • Issued: 03/12/2019
  • Est. Priority Date: 06/19/2015
  • Status: Active Grant
First Claim
Patent Images

1. A system for classifying information, the system comprising:

  • at least one computing device having a processor and physical memory, the physical memory storing instructions that cause the processor to;

    receive an identification of a relevant document;

    select a first set of documents from a document collection, wherein the document collection is stored on a non-transitory storage medium;

    assign a first set of default classifications to documents in the first set of documents to be used as a training set along with the relevant document;

    train a classifier using the training set;

    score one or more documents in the document collection using the classifier;

    upon determining that a stopping criteria has been reached, classify one or more documents in the document collection using the classifier;

    upon determining that a stopping criteria has not been reached, select a second set of documents having a batch size for presenting to a reviewer for review prior to repeating the step of training the classifier;

    present one or more documents in the second set of documents to the reviewer;

    receive from the reviewer user coding decisions associated with the presented documents;

    add one or more of the documents presented to the reviewer for which user coding decisions were received to the training set;

    remove one or more documents in the first set of documents from the training set;

    add a third set of documents from the document collection to the training set;

    assign a second set of default classifications to one or more documents in the third set of documents;

    update the classifier using one or more documents in the training set;

    increase the batch size of documents selected for the second set of documents; and

    repeat the steps of training, scoring and determining whether a stopping criteria has been reached;

    wherein the first and second set of default classifications are presumptively assigned classifications used for the purpose of training the classifier in order to form a decision boundary, the presumptively assigned classifications not being based on a review; and

    wherein the one or more documents in the first set of documents removed from the training set are documents previously assigned a presumptively assigned classification.

View all claims
  • 0 Assignments
Timeline View
Assignment View
    ×
    ×