AUTOMATED COLLECTION OF HUMAN-REVIEWED DATA
First Claim
1. A method of automated collection of human-reviewed data (HRD), comprising:
- receiving a data request from a requester by a data processing system, wherein the data processing system defines a task design component, a task dispatcher component, a result poller component and a result analyzer component;
transforming the data request into one or more human intelligence tasks (HITs) with the assistance of the task design component of the data processing system, wherein each HIT is specific to a respective HRD collection system;
sending each HIT to the respective HRD collection system by using the task dispatcher component;
collecting the HRD from each HRD collection system with the assistance of the result poller component, wherein the HRD is provided by an answerer based on each HIT;
analyzing the collected HRD with the assistance of the analyzer component;
wherein the analysis improves the accuracy of the HRD; and
sending the analyzed collected HRD to the requester.
3 Assignments
0 Petitions
Accused Products
Abstract
The embodiments of the present invention provide methods and systems for automated collection of human-reviewed data. Requesters send data to be reviewed by humans (or data requests) to a data processing system, which is in communication with one or more systems for collecting human-reviewed data (HRD). The methods and systems discussed enables the data processing system to work with one or more of the systems for collecting HRD). In one embodiment, between the data processing system and the systems for collecting HRD are wrappers, which stores parameters specific to the data requests and libraries for transforming the data requests to human intelligent tasks (HITs) specific to each HRD system. The data processing system also includes a number of components that facilitate transforming data requests into HITs, sending the HITs to the HRD collection systems, receiving HRD, and analyzing HRD to improve the quality of collected HRD.
-
Citations
25 Claims
-
1. A method of automated collection of human-reviewed data (HRD), comprising:
-
receiving a data request from a requester by a data processing system, wherein the data processing system defines a task design component, a task dispatcher component, a result poller component and a result analyzer component; transforming the data request into one or more human intelligence tasks (HITs) with the assistance of the task design component of the data processing system, wherein each HIT is specific to a respective HRD collection system; sending each HIT to the respective HRD collection system by using the task dispatcher component; collecting the HRD from each HRD collection system with the assistance of the result poller component, wherein the HRD is provided by an answerer based on each HIT; analyzing the collected HRD with the assistance of the analyzer component; wherein the analysis improves the accuracy of the HRD; and sending the analyzed collected HRD to the requester. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A system for automated collection of human-reviewed data (HRD), comprising:
-
a data processing system for receiving data request from a requester; an HRD collection system for collecting HRD corresponding to a human intelligence task (HIT) generated from the data request, wherein the HRD collected are entered by an answerer interacting with the HRD collection system; and a system with a wrapper between the data processing system and the HRD collection system, wherein the wrapper and the data processing system transform the received data request into the HIT to be sent to the HRD collection system for the answerer to view to prepare the HRD corresponding to the data request, and wherein the wrapper and the data processing system analyze the collected HRD to improve the accuracy of the HRD collected. - View Dependent Claims (16, 17, 18, 19, 20, 21)
-
-
22. Computer readable media including program instructions for automated collection of human-reviewed data (HRD), comprising:
-
program instructions for receiving a data request from a requester by a data processing system, wherein the data processing system defines a task design component, a task dispatcher component, a result poller component and a result analyzer component; program instructions for transforming the data request into one or more human intelligence tasks (HITs) with the assistance of the task design component of the data processing system, wherein each HIT is specific to a respective HRD collection system; program instructions for sending each HIT to the respective HRD collection system by using the task dispatcher component; program instructions for collecting the HRD from each HRD collection system with the assistance of the result poller component, wherein the HRD is provided by an answerer based on each HIT; program instructions for analyzing the collected HRD with the assistance of the analyzer component;
wherein the analysis improves the accuracy of the HRD collected; andprogram instructions for sending the analyzed collected HRD to the requester. - View Dependent Claims (23, 24, 25)
-
Specification