Interactive learning-based document annotation
First Claim
1. A document annotation system comprising:
- a graphical user interface for annotating documents, the graphical user interface including at least one user input device and a display device configured to display documents;
a probabilistic active learning component configured to train an annotation model and to propose annotations to documents based on the annotation model, the probabilistic active learning component also outputting a probability of acceptance associated with each proposed annotation; and
a request handler configured to convey annotation requests from the graphical user interface to the active learning component and to convey proposed annotations from the active learning component to the graphical user interface, the request handler including a mode selector that selects at least between (i) a training mode in which low probability proposed annotations are presented by the graphical user interface and (ii) an annotation mode in which high probability proposed annotations are presented by the graphical user interface.
6 Assignments
0 Petitions
Accused Products
Abstract
A document annotation system includes a graphical user interface used by an annotator to annotate documents. An active learning component trains an annotation model and proposes annotations to documents based on the annotation model. A request handler conveys annotation requests from the graphical user interface to the active learning component, conveys proposed annotations from the active learning component to the graphical user interface, and selectably conveys evaluation requests from the graphical user interface to a domain expert. During annotation, at least some low probability proposed annotations are presented to the annotator by the graphical user interface. The presented low probability proposed annotations enhance training of the annotation model by the active learning component.
-
Citations
11 Claims
-
1. A document annotation system comprising:
-
a graphical user interface for annotating documents, the graphical user interface including at least one user input device and a display device configured to display documents; a probabilistic active learning component configured to train an annotation model and to propose annotations to documents based on the annotation model, the probabilistic active learning component also outputting a probability of acceptance associated with each proposed annotation; and a request handler configured to convey annotation requests from the graphical user interface to the active learning component and to convey proposed annotations from the active learning component to the graphical user interface, the request handler including a mode selector that selects at least between (i) a training mode in which low probability proposed annotations are presented by the graphical user interface and (ii) an annotation mode in which high probability proposed annotations are presented by the graphical user interface. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A document annotation system comprising:
-
a graphical user interface for annotating documents, the graphical user interface including at least one user input device and a display device configured to display documents; an active learning component for training an annotation model and for proposing annotations to documents based on the annotation model, the active learning component comprising a probabilistic active learning component that outputs a probability of acceptance associated with each proposed annotation; and an asynchronous request handler configured to convey annotation requests from the graphical user interface to the active learning component and to convey proposed annotations from the active learning component to the graphical user interface, the asynchronous request handler (i) buffering annotation requests conveyed from the graphical user interface to the active learning component and (ii) buffering proposed annotations to documents conveyed from the active learning component to the graphical user interface, wherein the asynchronous request handler comprises a mode selector that selects at least between (i) a training mode in which low probability proposed annotations are presented by the graphical user interface and (ii) an annotation mode in which high probability proposed annotations are presented by the graphical user interface. - View Dependent Claims (9, 10, 11)
-
Specification