WEIGHT BENEFIT EVALUATOR FOR TRAINING DATA

• US 20150206065A1
• Filed: 08/05/2014
• Published: 07/23/2015
• Est. Priority Date: 11/22/2013
• Status: Active Grant

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: ABU-MOSTAFA, YASER SAID, GONZALEZ, CARLOS ROBERTO
• Granted Patent: US 10,558,935 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Technologies are generally described for methods and systems effective to determine a weight benefit associated with application of weights to training data in a machine learning environment. In an example, a device may determine a first function based on the training data, where the training data includes training inputs and training labels. The device may determine a second function based on weighted training data, which is based on application of weights to the training data. The device may determine a third function based on target data, where the target data is generated based on a target function. The target data may include target labels different from the training labels. The device may determine a fourth function based on weighted target data, which is a result of application of weights to the target data. The device may determine the weight benefit based on the first, second, third, and fourth functions.

Claims

1. A method to determine a weight benefit associated with training data in a machine learning environment, the method comprising, by a device:

determining a first function based on the training data, wherein the training data includes training inputs and training labels;

applying a set of weights to the training data to generate weighted training data;

determining a second function based on the weighted training data;

generating target data based on a target function, wherein the target data includes target labels different from the training labels;

determining a third function based on the target data;

applying the set of weights to the target data to generate weighted target data;

determining a fourth function based on weighted target data; and

determining the weight benefit based on the first, second, third, and fourth functions, wherein the weight benefit is associated with a benefit to apply the set of weights to the training data.

2. The method of claim 1, further comprising:

determining a first expected value between the first function and the second function;

determining a second expected value between the third function and the target function, wherein the target function is based on the training data;

determining a third expected value between the fourth function and the target function; and

determining a fourth expected value between the third function and the fourth function,

wherein determination of the weight benefit is based on the first, second, third, and fourth expected values.

3. The method of claim 2, wherein the first, second, third, and fourth expected values are based on an application of test inputs to the first, second, third, and fourth functions, respectively.

4. The method of claim 2, further comprising:

determining an evaluation value with use of the second, third, and fourth expected values;

comparing the evaluation value with the first expected value;

determining a count based on the comparison of the evaluation value with the first expected value; and

comparing the count with a threshold,

wherein the determination of the weight benefit is further based on the comparison of the count with the threshold.

5. The method of claim 2, wherein generation of the weighted target data is based on the target function.

6. The method of claim 1, further comprising:

determining a set of parameters to generate an artificial function;

generating artificial data with use of the training inputs and the artificial function, wherein the artificial data includes the training inputs and artificial labels; and

generating the target function based on the training data and the artificial data;

determining a first expected value between the first function and the second function;

determining a second expected value between the third function and the target function;

determining a third expected value between the fourth function and the target function; and

determining a fourth expected value between the third function and the fourth function,

wherein determination of the weight benefit is based on the first, second, third, and fourth expected values.

7. The method of claim 6, wherein generation of the target function comprises:

determining a first mean associated with the training data;

determining a first standard deviation associated with the training data;

determining a second mean associated with the artificial data;

determining a second standard deviation associated with the artificial data; and

wherein the target function is based on the first mean, the first standard deviation, the second mean, and the second deviation.

8. The method of claim 7, wherein the target function is further based on a noise parameter.

9. The method of claim 7, wherein the target function is based on an application of one of:

a sign function to the first mean, the first standard deviation, the second mean, and the second standard deviation;

a round function to the first mean, the first standard deviation, the second mean, and the second standard deviation;

or

a nearest number function to the first mean, the first standard deviation, the second mean, and the second standard deviation.

10. A system effective to determine a weight benefit associated with training data in a machine learning environment, the system comprising:

a memory configured to;

store the training data, wherein the training data includes training inputs and training labels; and

store a set of weights;

a target function generation module configured to be in communication with the memory, the target function generation module being configured to;

generate target data based on a target function, wherein the target data includes target labels different from the training labels; and

store the target data in the memory;

a processing module configured to be in communication with the target function generation module and the memory, the processing module being configured to;

apply the set of weights to the training data to generate weighted training data;

apply the set of weights to the target data to generate weighted target data;

store the weighted training data and the weighted target data in the memory;

a machine learning module configured to be in communication with the target function generation module, the processing module, and the memory, the machine learning module being configured to;

determine a first function based on the training data;

determine a second function based on the weighted training data

determine a third function based on the target data; and

determine a fourth function based on weighted target data; and

an evaluation module configured to be in communication with the target function generation module, the processing module, the machine learning module, and the memory, the evaluation module being configured to determine the weight benefit based on the first, second, third, and fourth functions, wherein the weight benefit is associated with a benefit to apply the set of weights to the training data.

11. The system of claim 10, further comprising an arithmetic module configured to be in communication with the target function generation module, the processing module, the machine learning module, the evaluation module, and the memory, the arithmetic module being configured to:

determine a first expected value between the first function and the second function;

retrieve the target function from the memory, wherein the target function is based on the training data;

determine a second expected value between the third function and the target function;

determine a third expected value between the fourth function and the target function;

determine a fourth expected value between the third function and the fourth function;

send the first, second, third, and fourth expected values to the evaluation module; and

the evaluation module being effective to determine the weight benefit based on the first, second, third, and fourth expected values.

12. The system of claim 11, wherein the evaluation module is further configured to:

receive the first, second, third, and fourth expected values;

determine an evaluation value with use of the second, third, and fourth expected values;

compare the evaluation value with the first expected value;

determine a count based on the comparison of the evaluation value with the first expected value; and

compare the count with a threshold,

wherein the determination of the weight benefit is further based on the comparison of the count with the threshold.

13. The system of claim 11, wherein the target function generation module is further configured to

generate the weighted target data with use of the target function.

14. The system of claim 10, wherein the target function generation module is further configured to:

determine a set of parameters to generate an artificial function;

generate artificial data with use of the training inputs and the artificial function, wherein the artificial data includes the training inputs and artificial labels;

generate the target function based on the training data and the artificial data;

determine a first expected value between the first function and the second function;

determine a second expected value between the third function and the target function;

determine a third expected value between the fourth function and the target function;

determine a fourth expected value between the third function and the fourth function; and

the evaluation module being effective to determine the weight benefit based on the first, second, third, and fourth expected values.

15. The system of claim 14, wherein the target function generation module is further configured to:

determine a first mean associated with the training data;

determine a first standard deviation associated with the training data;

determine a second mean associated with the artificial data; and

determine a second standard deviation associated with the artificial data,

wherein the target function is based on the artificial data, the first mean, the first standard deviation, the second mean, and the second deviation.

16. A method to determine a weight benefit associated with training data in a machine learning environment, the method comprising, by a first device:

receiving, from a second device, a first function that is based on the training data, where the training data includes training inputs and training labels;

receiving, from the second device, a set of weights;

receiving, from the second device, a second function that is based on weighted training data, wherein the weighted training data is based on the set of weights;

generating target data based on a target function, wherein the target data includes target labels different from the training labels;

determining a third function based on the target data;

applying the set of weights to the target data to generate weighted target data;

determining a fourth function based on weighted target data; and

determining the weight benefit based on the first, second, third, and fourth functions, wherein the weight benefit is associated with a benefit to apply the set of weights to the training data.

17. The method of claim 16, further comprising:

determining a first expected value between the first function and the second function;

determining a second expected value between the third function and the target function;

determining a third expected value between the fourth function and the target function; and

determining a fourth expected value between the third function and the fourth function,

wherein determination of the weight benefit is based on the first, second, third, and fourth expected values.

18. The method of claim 17, wherein the first, second, third, and fourth expected values are based on an application of test inputs to the first, second, third, and fourth functions, respectively.

19. The method of claim 17, further comprising:

determining an evaluation value with use of the second, third, and fourth expected values;

comparing the evaluation value with the first expected value;

determining a count based on the comparison of the evaluation value with the first expected value; and

comparing the count with a threshold,

wherein the determination of the weight benefit is further based on the comparison of the count with the threshold.

20. The method of claim 16, further comprising:

determining a set of parameters to generate an artificial function;

generating artificial data with use of the training inputs and the artificial function, wherein the artificial data includes the training inputs and artificial labels;

determining a first mean associated with the training data;

determining a first standard deviation associated with the training data;

determining a second mean associated with the artificial data;

determining a second standard deviation associated with the artificial data; and

generating the target function with use of the artificial data, the first mean, the first standard deviation, the second mean, and the second standard deviation.