TASKING SYSTEM

• US 20170323233A1
• Filed: 07/25/2016
• Published: 11/09/2017
• Est. Priority Date: 05/09/2016
• Status: Active Grant

• Current Assignee: Aurora Operations Incorporated (Aurora Innovation, Inc.)
• Original Assignee: Mighty Corporation (Japan Tobacco Inc.)
• Inventors: Bencke, Matthew Justin Von, Hugeback, Angela Beth, Li, Yuan, Nakhuda, Daryn Edward, O'Donnell, Patrick Emmett, Shobe, Matthew Newman
• Granted Patent: US 10,733,556 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/0633 Workflow analysis
• CPC Class Codes: H04L 67/306 User profiles

Abstract

Disclosed is a tasking system to assign individuals to tasks. The tasking system targets a user for the task based information in a user profile such as demographics, background and reputation of the user to perform a task. The tasking system predicts if a user will do a given task well based on a user'"'"'s background, user'"'"'s skill, and applied predictive algorithms. The tasking system manages the workflow of the task, assesses the probability of the user'"'"'s answer being accurate, and continuously optimizes assignment and assessment of the task to improve data quality.

Claims

1. A method determining computerized tasks for assignment to a user of a plurality of users, the method comprising:

targeting a user to perform a task based on targeting criteria and stored information from a user profile;

predicting success of the user to perform the task based on a past performance of the user on a similar task;

assigning the task to the user if the predicted success is greater than a threshold level;

managing a workflow for the task assigned to the user;

assessing if the task as completed by the user is within a predefined acceptance range; and

receiving customer feedback on the task when completed by the user.

2. The method of claim 1, further comprising:

receiving a login from a user through an external system; and

storing external user information in the user profile of the user based on information from the external system.

3. The method of claim 1, wherein the stored information in a user profile further comprises information on a reputation of the user based on previous performance and activities of the user.

4. The method of claim 1, wherein the stored information in a user profile further comprises information on a reputation of the user based on previous performance and activities of the user for a given domain.

5. The method of claim 1, wherein the stored information in a user profile further comprises information on a reputation of the user based on previous performance and activities of the user for a given task type.

6. The method of claim 1, wherein predicting the success of the user is further based on at least one of the following:

patterns in previous tasking behavior of the user, user data from outside sources, and user surveys.

7. The method of claim 1, wherein assigning the task further comprises:

training the user for a task set, qualifying the user to perform the task set, and giving a user access to the task after the user successfully completes the training and the qualifying steps for the task set.

8. The method of claim 7, wherein assigning the task further comprises:

adjusting the threshold level for assigning the task to the user based on a total number of qualified users identified through the training and the qualifying steps.

9. The method of claim 1, wherein assigning the task further comprises:

setting a maximum number of tasks an individual can complete.

10. The method of claim 1, wherein managing the workflow for the task assigned to the user further comprises varying at least one of an assignment, qualification, and ongoing quality criteria at each step of the workflow.

11. The method of claim 1, wherein managing the workflow for the task assigned to the user further comprises manipulating the workflow to determine a tradeoff among velocity, quality, and cost.

12. The method of claim 1, wherein the predefined acceptance range is based on at least one of:

answers from a large number of users, answers from users weighted by user reputation, a known answer, and assessment of the completed task by a mentor, a mentor being a user with elevated status.

13. The method of claim 12, wherein the mentor is a user who has received positive feedback on completed tasks greater than a positive feedback threshold.

14. The method of claim 1, further comprising:

identifying common characteristics of users based on previous performance of the users on a given set of tasks being equal to or above an acquisition threshold value;

acquire additional users with the common characteristics; and

adjust the common characteristics, the acquisition threshold value, spend, and acquisition velocity of additional users based on assessment of additional tasks as completed by one or more of the additional users.

15. A non-transitory computer-readable storage medium comprising stored executable computer program instructions for assigning tasks to users, the computer program instructions when executed by a computer processor cause the computer processor to perform steps comprising:

target a user to perform a task based on targeting criteria and stored information from a user profile;

predict success of the user to perform the task based on a past performance of the user on a similar task;

assign the task to the user if the predicted success is greater than a threshold level;

manage a workflow for the task assigned to the user;

assess if the task as completed by the user is within a predefined acceptance range; and

receive customer feedback on the task when completed by the user.

16. The computer readable medium of claim 15, further comprising instructions that cause the processor to:

receive a login from a user through an external system; and

store external information in the user profile of the user based on information from the external system.

17. The computer readable medium of claim 15, wherein the stored information in a user profile further comprises information on a reputation of the user based on previous performance and activities of the user.

18. The computer readable medium of claim 15, wherein the prediction of success of the user to perform the task is further based on at least one of the following:

patterns in previous tasking behavior of the user, user data from outside sources, and user surveys.

19. The computer readable medium of claim 15, wherein the instruction to assign the task further comprises instructions that cause the processor to:

train the user for a task set, qualify the user to perform the task set, and give a user access to the task after the user successfully completes the training and the qualifying steps for the task set.

20. The computer readable medium of claim 19, wherein the instruction to assign the task further comprises instructions that cause the processor to:

adjust the threshold level for assigning the task to the user based on a total number of qualified users identified through the training and the qualifying steps.

21. The computer readable medium of claim 15, wherein the instruction to assign the task further comprises instructions that cause the processor to:

set a maximum number of tasks an individual can complete.

22. The computer readable medium of claim 15, wherein the instruction to manage the workflow of the task assigned to the user further comprises instructions to vary at least one of an assignment, qualification, and ongoing quality criteria at each step of the workflow.

23. The computer readable medium of claim 15, wherein the predefined acceptance range is based on at least one of:

answers from a large number of users, answers from users weighted by user reputation, a known answer, and assessment of the completed task by a mentor, a mentor being a user with elevated status.

24. The computer readable medium of claim 23, wherein the mentor is a user who has received positive feedback on completed tasks greater than a positive feedback threshold.

25. A non-transitory computer-readable storage medium comprising stored executable computer program instructions for assigning tasks to users, the computer program instructions when executed by a computer processor cause the computer processor to perform steps comprising:

target a user to perform a task based on targeting criteria and stored information from a user profile;

predict success of the user to perform the task based on a past performance of the user on a similar task;

assign the task to the user if the predicted success is greater than a threshold level;

manage a workflow for the task assigned to the user;

assess the answer quality of a task as completed by the user using a model, wherein the model is trained based on marked answer data collected from current task set or similar task sets, wherein answers on marked answer data is determined based at least one of;

community quality assurance, consensus analysis, and heuristics and telemetry data of users; and

receive customer feedback on the task when completed by the user.