×

System and method for incremental annotation of datasets

  • US 10,496,369 B2
  • Filed: 07/30/2018
  • Issued: 12/03/2019
  • Est. Priority Date: 07/31/2017
  • Status: Active Grant
First Claim
Patent Images

1. A system for incremental annotation of datasets, the system comprising:

  • at least one storage device configured to store a plurality of labeled examples and a plurality of unlabeled examples; and

    at least one processor configured to;

    use the plurality of labeled examples to generate a first inference model;

    use the first inference model to assign labels to at least part of the unlabeled examples;

    calculate confidence levels corresponding to the assigned labels;

    use the confidence levels to select a subset of the plurality of unlabeled examples, where at least one of the plurality of unlabeled examples is not included in the selected subset of the plurality of unlabeled examples;

    generate a second inference model based on the plurality of labeled examples, the selected subset of the plurality of unlabeled examples, and the assigned labels corresponding to the selected subset of the plurality of unlabeled examples;

    use the confidence levels to select a user of a plurality of alternative users;

    provide a request to the selected user to assign labels;

    in response to the request, receive from the user an assignment of labels to one or more of the unlabeled examples; and

    generate a third inference model based on the plurality of labeled examples, the one or more of the unlabeled examples, and the assignment of labels received from the user.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×