Automated validation of results of human performance of tasks
First Claim
1. A computer-implemented method for providing an electronic marketplace that acts as an intermediary to facilitate performance by task performer users of tasks supplied by task requesters, the method comprising:
- for each of multiple tasks, receiving information about the task from a task requester user who is submitting the task as being available for performance by one or more task performer users, at least some of the multiple tasks each having one or more associated validation tests that are specified by the task requester user who submitted the task, the one or more associated validation tests for one of the at least some tasks including obtaining initial results from one or more initial task performer users and receiving an evaluation of the initial results from one or more other task performer users that are more highly qualified than the initial task performer users, such that success of those one or more associated validation tests includes that the evaluation is positive;
automatically facilitating performance of at least some of the multiple submitted tasks by providing information about tasks to task performer users and receiving results of performance of tasks by task performer users, the received results including results for one or more of the at least some tasks, the automatic facilitating being performed by one or more configured computing systems that are part of the electronic marketplace; and
for each of the one or more of the at least some tasks, after the receiving of the results of performance of the task by one or more task performer users,automatically performing the one or more associated validation tests on the results for the task and determining that the results are acceptable based at least in part on the performed validation tests succeeding, the automatic performing of the one or more associated validation tests being performed by the one or more configured computing systems; and
in response to determining that the results are acceptable, providing the results to the task requester user who submitted the task.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques are described for facilitating automated validation of results of performance of tasks. In some situations, task requesters may specify various automated validation criteria for tasks that they submit, and those specified validation criteria may automatically be used to determine whether results of performance of those tasks by task performers are validated. To facilitate such automated validation, predefined types of validation criteria may be provided that a user may select and/or configure. The outcome of automated validation of results may be used in various ways, such as to indicate when results have been validated and/or have not been validated, to determine whether to provide results to a task requester (e.g., if results are not provided unless they have been validated, or if results that fail validation are corrected or replaced with results that pass validation) and/or to compensate task performers for providing the results.
-
Citations
48 Claims
-
1. A computer-implemented method for providing an electronic marketplace that acts as an intermediary to facilitate performance by task performer users of tasks supplied by task requesters, the method comprising:
-
for each of multiple tasks, receiving information about the task from a task requester user who is submitting the task as being available for performance by one or more task performer users, at least some of the multiple tasks each having one or more associated validation tests that are specified by the task requester user who submitted the task, the one or more associated validation tests for one of the at least some tasks including obtaining initial results from one or more initial task performer users and receiving an evaluation of the initial results from one or more other task performer users that are more highly qualified than the initial task performer users, such that success of those one or more associated validation tests includes that the evaluation is positive; automatically facilitating performance of at least some of the multiple submitted tasks by providing information about tasks to task performer users and receiving results of performance of tasks by task performer users, the received results including results for one or more of the at least some tasks, the automatic facilitating being performed by one or more configured computing systems that are part of the electronic marketplace; and for each of the one or more of the at least some tasks, after the receiving of the results of performance of the task by one or more task performer users, automatically performing the one or more associated validation tests on the results for the task and determining that the results are acceptable based at least in part on the performed validation tests succeeding, the automatic performing of the one or more associated validation tests being performed by the one or more configured computing systems; and in response to determining that the results are acceptable, providing the results to the task requester user who submitted the task. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A computer-implemented method comprising:
-
receiving information from a task requester about one or more submitted tasks that are each available for performance by one or more task performers and that each have one or more associated validation tests, the one or more validation tests associated with one of the submitted tasks being specified by the task requester who submitted the one task, the specifying being via interactions with a user interface provided by one or more configured computing systems acting as an intermediary between task requester users and task performer users to facilitate performance of tasks; and for each of at least one of the submitted tasks, receiving one or more results from performance of the task by one or more task performers; automatically applying at least one of the validation tests associated with the task to the received one or more results, the automatic applying being performed by the one or more configured computing systems; automatically determining that the received one or more results are validated based at least in part on the received one or more results passing the applied validation tests, the automatic determining being performed by the one or more configured computing systems; and providing the received one or more results to the task requester. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26)
-
-
27. A computer-implemented method comprising:
-
receiving information from a task requester about one or more submitted tasks that are each available for performance by one or more task performers and that each have one or more associated validation tests, wherein one of the submitted tasks further has one or more associated validation criteria other than the one or more validation tests associated with that one task; and for each of at least one of the submitted tasks, receiving one or more results from performance of the task by one or more task performers; automatically applying at least one of the validation tests associated with the task to the received one or more results, the automatic applying being performed by one or more configured computing systems; automatically determining that the received one or more results are validated based at least in part on the received one or more results passing the applied validation tests, the automatic determining being performed by the one or more configured computing systems, and wherein the determining is further based at least in part on the received one or more results for the one task satisfying the one or more associated validation criteria; and providing the received one or more results to the task requester.
-
-
28. A computer-implemented method comprising:
-
receiving information from a task requester about one or more submitted tasks that are each available for performance by one or more task performers and that each have one or more associated validation tests; and for each of at least one of the submitted tasks, receiving one or more results from performance of the task by one or more task performers; automatically applying at least one of the validation tests associated with the task to the received one or more results, the automatic applying being performed by one or more configured computing systems; automatically determining that the received one or more results are validated based at least in part on the received one or more results passing the applied validation tests, the automatic determining being performed by the one or more configured computing systems; and providing the received one or more results to the task requester, and wherein one of the validation tests for one of the submitted tasks includes obtaining a verification of one or more results received from performance of the one task by one or more task performers, the verification being from one or more other task performers that did not provide the one or more received results. - View Dependent Claims (29, 30)
-
-
31. A computer-implemented method comprising:
-
receiving information from a task requester about one or more submitted tasks that are each available for performance by one or more task performers and that each have one or more associated validation tests, wherein one of the submitted tasks has multiple associated validation tests and further includes one or more instructions for use of the multiple associated validation tests; and for each of at least one of the submitted tasks, receiving one or more results from performance of the task by one or more task performers; automatically applying at least one of the validation tests associated with the task to the received one or more results, the automatic applying being performed by one or more configured computing systems, and wherein the applying of the at least one validation tests associated with the one task having the multiple associated validation tests includes applying one or more of the multiple associated validation tests in accordance with the included instructions; automatically determining that the received one or more results are validated based at least in part on the received one or more results passing the applied validation tests, the automatic determining being performed by the one or more configured computing systems; and providing the received one or more results to the task requester. - View Dependent Claims (32, 33, 34)
-
-
35. A computer-implemented method comprising:
-
presenting a user interface to users that provides information about multiple predefined types of validation tests and that provides capabilities for the users to interactively select and configure one or more of the validation test types for one or more tasks; receiving information from a task requester about one or more submitted tasks that are each available for performance by one or more task performers and that each have one or more associated validation tests, and wherein at least one of the validation tests for at least one of the submitted tasks is specified based on interactions of a task requester user with the presented user interface; and for each of at least one of the submitted tasks, receiving one or more results from performance of the task by one or more task performers; automatically applying at least one of the validation tests associated with the task to the received one or more results, the automatic applying being performed by one or more configured computing systems; automatically determining that the received one or more results are validated based at least in part on the received one or more results passing the applied validation tests, the automatic determining being performed by the one or more configured computing systems; and providing the received one or more results to the task requester.
-
-
36. A non-transitory computer-readable medium whose contents configure a computing device to automatically validate results of task performance, by performing a method comprising:
-
receiving an indication from a user of one or more tasks that are not yet performed, the receiving of the indication of the one or more tasks including receiving an indication of a type of task such that the one or more tasks are of the indicated task type; providing information to the user to indicate at least one predefined type of validation criteria that is available; and after receiving an indication from the user to use one or more of the at least one predefined types of validation criteria in an indicated manner, using the indicated one or more predefined types of validation criteria in the indicated manner to automatically validate results of performance of the one or more tasks, the using of the indicated one or more predefined types of validation criteria including storing the indicated one or more predefined types of validation criteria for tasks of the indicated task type that are later submitted by the user. - View Dependent Claims (37, 38, 39, 40)
-
-
41. A non-transitory computer-readable medium whose contents configure a computing device to automatically validate results of task performance, by performing a method comprising:
-
receiving an indication from a user of one or more tasks that are not yet performed; providing information to the user to indicate at least one predefined type of validation criteria that is available; and after receiving an indication from the user to use one or more of the at least one predefined types of validation criteria in an indicated manner, using the indicated one or more predefined types of validation criteria in the indicated manner to automatically validate results of performance of the one or more tasks, and wherein the receiving of the indication from the user to use the one or more predefined types of validation criteria in the indicated manner includes defining one or more validation criteria that are each of one or more of the indicated at least one predefined types, and wherein the using of the indicated one or more predefined types of validation criteria includes, after receiving one or more results from performance of the task by one or more task performers, automatically determining whether the received one or more results are validated based at least in part on whether the received one or more results satisfy the one or more defined validation criteria. - View Dependent Claims (42, 43)
-
-
44. A computing device configured to automatically validate results of task performance, comprising:
-
one or more processors; a first component that consists of a means for receiving information about a task that is available from a task requester and that has one or more associated validation criteria, the first component being configured to, when executed by at least one of the one or more processors, receive the information about the task that is available from the task requester and that has the one or more associated validation criteria; and a second component that consists of a means for, after receiving one or more results from performance of the task by one or more task performers, automatically determining whether the received one or more results are validated based at least in part on whether the received one or more results satisfy the one or more associated validation criteria for the task, the second component being configured to, when executed by at least one of the one or more processors, and after the receiving of the one or more results from the performance of the task by the one or more task performers, automatically perform the determining of whether the received one or more results are validated based at least in part on whether the received one or more results satisfy the one or more associated validation criteria for the task. - View Dependent Claims (45, 46, 47, 48)
-
Specification