MACHINE LEARNING SYSTEM FOR A TASK BROKERAGE SYSTEM

• US 20090240539A1
• Filed: 03/21/2008
• Published: 09/24/2009
• Est. Priority Date: 03/21/2008
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Dendi, Vikram, Chandrasekar, Raman, Slawson, Dean A.
• Application Number:
  US12/053,259
• Publication Number:
  US 20090240539A1
• Time in Patent Office: Days
• US Class Current: 705/7
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data

Abstract

A machine learning system learns models to assist providers in processing documents of customers. Providers may use various productivity tools that use the learned models to assist in performing tasks on target documents of customers. The machine learning system may initially train models based on demographic information of customers and training data of the customers. To generate the models, the machine learning system collects the training data for the customers of each cluster and then trains a model for each cluster. The machine learning system uses the models to perform tasks on documents of customers. A provider can then modify the results of the task. The machine learning system can use those modifications to adjust the models.

Claims

1. A method in a computing device for learning models for processing documents in a task brokerage system, the method comprising:

providing demographic information for customers;

providing training data for the customers;

generating models based on the provided training data and demographic information of the customers;

receiving a target document of a customer;

selecting a generated model based on demographic information of the customer;

applying the selected model to the target document to generate a result;

determining refinements to the result made by a provider when generating a refined result; and

adjusting the selected model based on the refinements to the result made by the provider

wherein refinements made by providers to results of applying a selected model to a target document are used to adjust the selected model.

2. The method of claim 1 wherein the generating of the models includes:

identifying clusters of customers based on their demographic information; and

for each cluster, training a model based on the training data of the customers within the cluster.

3. The method of claim 1 including combining models when a distance between models is less than a combine threshold distance.

4. The method of claim 3 wherein models are combined by training a combined model using training data and refinements of results of target documents of customers within the clusters of the models to be combined.

5. The method of claim 1 including splitting a model for a cluster when models generated for sub-clusters of customers of the cluster have a distance that is greater than a split threshold distance.

6. The method of claim 5 wherein the splitting includes:

identifying sub-clusters of customers of the cluster; and

for each sub-cluster, generating a model using training data and refinements of results of target documents of customers of the sub-cluster.

7. The method of claim 1 wherein the processing of documents includes language translation of the target document of a first language into the result in a second language.

8. The method of claim 1 wherein the generated model is further selected based on the task and related information.

9. The method of claim 1 wherein the refinements are determined by collecting corrections the provider makes to the result.

10. The method of claim 1 wherein the refinements are determined by identifying differences between the result and the refined result.

11. The method of claim 1 wherein the applying of the selected model to the target document is performed at a server and the result is sent to the provider wherein the provider cannot download the model.

12. The method of claim 1 wherein the selecting of the generated model is further based on input from a provider.

13. The method of claim 1 wherein the models are learned based on training data of customers of a group of providers.

14. A computing device for providing models for processing documents of customers in a task brokerage system, comprising:

a model store containing models for processing documents, the models being learned based on training data and demographic information of customers of the task brokerage system;

a component that selects a model for a customer and applies the selected model to a target document of a customer to generate a result;

a component that identifies refinements to the result made by a provider when generating a refined result for the result; and

a component that adjusts the selected model based on the refinements to the result made by the provider to the result.

15. The computing device of claim 14 wherein the models are learned by identifying clusters of customers based on their demographic information and, for each cluster, training a model based on the training data of the customers within the cluster.

16. The computing device of claim 14 including a component that combines models when a distance between models is less than a combine threshold distance.

17. The computing device of claim 14 including a component that splits a model for a cluster when models generated for sub-clusters of customers of the cluster have a distance that is greater than a split threshold distance.

18. The computing device of claim 14 including a component that inputs a selection of customers from a provider and generates a model for the provider based on training data of the selected customers.

19. The computing device of claim 14 including a component that recommends a provider to a customer based on analysis of performance of the providers on target documents of customers with similar demographic information to the customer.

20. A computer-readable storage medium encoded with computer-executable instructions for learning models for processing documents in a task brokerage system, by a method comprising:

providing demographic information for customers;

providing training data for the customers;

generating models by identifying clusters of customers based on their demographic information and, for each cluster, training a model based on the training data of the customers within the cluster;

for each of a plurality of target documents of customers, receiving the target document of a customer;

selecting a generated model based on demographics of the customer;

applying the selected model to the target document to generate a result;

providing the result to a provider for refinement; and

identifying refinements to the result made by the provider; and

adjusting the models of the clusters based on the identified refinements made by the providers to the results of target documents of customers of the cluster so that the adjusted models can subsequently be applied to target documents of customers.