Alternative training distribution data in machine learning

• US 10,535,014 B2
• Filed: 08/05/2014
• Issued: 01/14/2020
• Est. Priority Date: 03/10/2014
• Status: Active Grant

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: Abu-Mostafa, Yaser Said, Gonzalez, Carlos Roberto
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Smith, Brian M
• Application Number:
  US14/451,935
• Publication Number:
  US 20150254573A1
• Time in Patent Office: 1,988 Days
• Field of Search: None
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound

Abstract

Technologies are generally described for methods and systems in a machine learning environment. In some examples, a method may include retrieving training data from a memory. The training data may include training inputs and training labels. The methods may further include determining a set of datasets based on the training inputs. The methods may further include determining a set of out of sample errors based on the training inputs and based on test data. Each out of sample error may correspond to a respective dataset in the set of datasets. The methods may further include generating alternative distribution data based on the set of out of sample errors. The alternative distribution data may be used to determine weights to be applied to the training data.

Claims

1. A method in a machine learning environment, the method comprising:

determining, by a processor of a computing device based on training data for a machine learning module, a set of possible datasets in an input space, wherein;

the set of possible datasets comprises a set of all

2. The method of claim 1, wherein determining the set of parameters of the objective function comprises applying a convex optimization technique to the objective function.

3. The method of claim 1, wherein determining the set of parameters includes determining parameters that reduce a value of the objective function.

4. The method of claim 1, further comprising generating weights for the training data based on the alternative distribution data.

5. The method of claim 1, wherein determining the set of out of sample errors comprises:

applying a transform to the training inputs to produce transformed inputs;

determining an error function based on the transformed inputs and based on test inputs for the machine learning module; and

determining the set of out of sample errors based on the error function.

6. The method of claim 5, wherein determining the error function includes combining a noise parameter with the transformed inputs and the test inputs to determine the error function.

7. A non-transitory computer-readable medium that includes computer-executable instructions stored thereon that are executable by the computing device of claim 1 to perform the method of claim 1.

8. The method of claim 1, wherein a number of datasets in the input space is calculated using:

9. A system effective to generate alternative distribution data in a machine learning environment, the system comprising:

a machine learning circuit;

a processor configured to be in communication with the machine learning circuit and further configured to determine, based on training data for the machine learning circuit, a set of possible datasets in an input space, wherein;

the set of possible datasets comprises a set of all

10. The system of claim 9, wherein the optimization circuit is configured to apply a convex optimization technique to the objective function to determine the set of parameters.

11. The system of claim 9, wherein the set of parameters includes parameters that reduce a value of the objective function.

12. The system of claim 9, wherein the processor is further configured to generate weights for the training data based on the alternative distribution data.

13. The system of claim 9, wherein to determine the set of out of sample errors, the evaluation circuit is configured to:

apply a transform to the training inputs to produce transformed inputs;

determine an error function based on the transformed inputs and based on test inputs for the machine learning circuit; and

determine the set of out of sample errors based on the error function.

14. The system of claim 13, wherein the error function comprises a noise parameter.

15. The system of claim 9, wherein a number of datasets in the input space is calculated by use of:

16. A method to generate alternative distribution data in a machine learning environment, the method comprising:

receiving, by a first device from a second device, training data for a machine learning module, wherein;

the training data comprises training inputs and training labels in an input space, and

the training data is associated with a learned function that corresponds to a first error rate;

determining, by a processor of the first device based on the training inputs, a set of possible datasets, wherein the set of possible datasets comprises a set of all

17. The method of claim 16, wherein determining the set of parameters comprises applying a convex optimization technique to the objective function.

18. The method of claim 16, wherein determining the set of parameters includes determining parameters that reduce a value of the objective function.

19. The method of claim 16, further comprising generating weights for the training data based on the alternative distribution data.

20. The method of claim 16, wherein determining the set of out of sample errors comprises:

applying a transform to the training inputs to produce transformed inputs;

determining an error function based on the transformed inputs and based on the test inputs; and

determining the set of out of sample errors based on the error function.

21. The method of claim 16, wherein a number of datasets is calculated using: