SYSTEMS AND METHODS FOR FEW-SHOT TRANSFER LEARNING

• US 20200130177A1
• Filed: 08/05/2019
• Published: 04/30/2020
• Est. Priority Date: 10/29/2018
• Status: Active Application

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Kolouri, Soheil, Rostami, Mohammad, Kim, Kyungnam
• Application Number:
  US16/532,321
• Publication Number:
  US 20200130177A1
• Time in Patent Office: Days
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 3/096 Transfer learning

Abstract

A method for training a controller to control a robotic system includes: receiving a neural network of an original controller for the robotic system based on origin data samples from an origin domain and labels in a label space, the neural network including encoder and classifier parameters, the neural network being trained to: map an input data sample from the origin domain to a feature vector in a feature space using the encoder parameters; and assign a label of the label space to the input data sample using the feature vector based on the classifier parameters; updating the encoder parameters to minimize a dissimilarity, in the feature space, between: origin feature vectors computed from the origin data samples; and target feature vectors computed from target data samples from a target domain; and updating the controller with the updated encoder parameters to control the robotic system in the target domain.

Claims

1. A method for training a controller to control a robotic system in a target domain, the method comprising:

receiving a neural network of an original controller for controlling the robotic system based on a plurality of origin data samples from an origin domain and corresponding labels in a label space the neural network of the original controller comprising a plurality of encoder parameters and a plurality of classifier parameters, the neural network being trained to;

map an input data sample from the origin domain to a feature vector in a feature space in accordance with the encoder parameters; and

assign a label of the label space to the input data sample based on the feature vector in accordance with the classifier parameters;

updating the encoder parameters to minimize a dissimilarity, in the feature space, between;

a plurality of origin feature vectors computed from the origin data samples; and

a plurality of target feature vectors computed from a plurality of target data samples from the target domain, the target data samples having a smaller cardinality than the origin data samples; and

updating the controller with the updated encoder parameters to control the robotic system in the target domain.

2. The method of claim 1, wherein the dissimilarity is computed in accordance with a sliced Wasserstein distance between the origin feature vectors in the feature space and the target feature vectors in the feature space.

3. The method of claim 1, wherein the updating the encoder parameters comprises iteratively computing a plurality of intermediate encoder parameters, each iteration comprising:

computing the origin feature vectors in the feature space;

computing the target feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing the dissimilarity between the origin feature vectors and the target feature vectors;

updating the intermediate encoder parameters to reduce the dissimilarity between the origin feature vectors and the target feature vectors;

determining whether the dissimilarity is minimized;

in response to determining that the dissimilarity is not minimized, proceeding with another iteration with the updated intermediate encoder parameters as the intermediate encoder parameters; and

in response to determining that the dissimilarity is minimized, outputting the intermediate encoder parameters as the updated encoder parameters.

4. The method of claim 3, wherein the dissimilarity is computed in accordance with a sliced Wasserstein distance between the origin feature vectors in the feature space and the target feature vectors in the feature space.

5. The method of claim 3, wherein the computing the origin feature vectors is performed by an origin encoder.

6. The method of claim 3, wherein the computing the origin feature vectors is performed in accordance with the intermediate encoder parameters.

7. The method of claim 1, wherein the target data samples comprise a plurality of target samples and a plurality of corresponding target labels.

8. The method of claim 1, wherein the target data samples comprise a plurality of unlabeled target samples.

9. The method of claim 8, wherein the updating the encoder parameters comprises iteratively computing a plurality of intermediate encoder parameters, each iteration comprising:

computing the origin feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing the target feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing predicted labels for the target feature vectors in accordance with the classifier parameters, each of the predicted labels being associated with a confidence;

defining a plurality of pseudo-labels corresponding to the predicted labels having confidences exceeding a threshold;

updating the intermediate encoder parameters based on at least one of;

minimizing a dissimilarity between the origin feature vectors and the target feature vectors; and

minimizing a classification loss of the origin data samples;

determining whether a stopping condition has been met, wherein the stopping condition comprises at least one of;

a dissimilarity between the origin feature vectors and the target feature vectors; and

a saturation of a number of the pseudo-labels between iterations;

in response to determining that the stopping condition has not been met, proceeding with another iteration with the updated intermediate encoder parameters as the intermediate encoder parameters; and

in response to determining that the stopping condition is met, outputting the intermediate encoder parameters as the updated encoder parameters.

10. The method of claim 9, wherein the updating the intermediate encoder parameters alternates between:

the minimizing the dissimilarity between the origin feature vectors and the target feature vectors; and

the minimizing the classification loss of the origin data samples.

11. The method of claim 1, wherein the neural network comprises a convolutional neural network, a recurrent neural network, a capsule network, or combinations thereof.

12. A system for training a controller to control a robotic system in a target domain, the system comprising:

a processor; and

non-volatile memory storing instructions that, when executed by the processor, cause the processor to;

receive a neural network of an original controller for controlling the robotic system based on a plurality of origin data samples from an origin domain and corresponding labels in a label space, the neural network of the original controller comprising a plurality of encoder parameters and a plurality of classifier parameters, the neural network being trained to;

map an input data sample from the origin domain to a feature vector in a feature space in accordance with the encoder parameters; and

assign a label of the label space to the input data sample based on the feature vector in accordance with the classifier parameters;

update the encoder parameters to minimize a dissimilarity between;

a plurality of origin feature vectors computed from the origin data samples; and

a plurality of target feature vectors computed from a plurality of target data samples from the target domain, the target data samples having a smaller cardinality than the origin data samples; and

update the controller with the updated encoder parameters to control the robotic system in the target domain.

13. The system of claim 12, wherein the dissimilarity is computed in accordance with a sliced Wasserstein distance between the origin feature vectors in the feature space and the target feature vectors in the feature space.

14. The system of claim 12, wherein the instructions that cause the processor to update the encoder parameters comprise instructions that, when executed by the processor cause the processor to iteratively compute a plurality of intermediate encoder parameters, each iteration comprising:

computing the origin feature vectors in the feature space;

computing the target feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing the dissimilarity between the origin feature vectors and the target feature vectors;

updating the intermediate encoder parameters to reduce the dissimilarity between the origin feature vectors and the target feature vectors;

determining whether the dissimilarity is minimized;

in response to determining that the dissimilarity is not minimized, proceeding with another iteration with the updated intermediate encoder parameters as the intermediate encoder parameters; and

in response to determining that the dissimilarity is minimized, outputting the intermediate encoder parameters as the updated encoder parameters.

15. The system of claim 12, wherein the target data samples comprise a plurality of target samples and a plurality of corresponding target labels.

16. The system of claim 12, wherein the target data samples comprise a plurality of unlabeled target samples.

17. The system of claim 16, wherein the instructions that cause the processor to update the encoder parameters comprise instructions that, when executed by the processor, cause the processor to compute the updated encoder parameters by iteratively computing a plurality of intermediate encoder parameters, each iteration comprising:

computing the origin feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing the target feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing predicted labels for the target feature vectors in accordance with the classifier parameters, each of the predicted labels being associated with a confidence;

defining a plurality of pseudo-labels corresponding to the predicted labels having confidences exceeding a threshold;

updating the intermediate encoder parameters based on at least one of;

minimizing a dissimilarity between the origin feature vectors and the target feature vectors; and

minimizing a classification loss of the origin data samples;

determining whether a stopping condition has been met, wherein the stopping condition comprises at least one of;

a dissimilarity between the origin feature vectors and the target feature vectors; and

a saturation of a number of the pseudo-labels between iterations;

in response to determining that the stopping condition has not been met, proceeding with another iteration with the updated intermediate encoder parameters as the intermediate encoder parameters; and

in response to determining that the stopping condition is met, outputting the intermediate encoder parameters as the updated encoder parameters.

18. The system of claim 17, wherein the updating the intermediate encoder parameters alternates between:

the minimizing the dissimilarity between the origin feature vectors and the target feature vectors; and

the minimizing a classification loss of the origin data samples.

19. The system of claim 12, wherein the neural network comprises a convolutional neural network, a recurrent neural network, a capsule network, or combinations thereof.

20. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processor, cause the processor to:

receive a neural network of an original controller for controlling a robotic system based on a plurality of origin data samples from an origin domain and corresponding labels in a label space, the neural network of the original controller comprising a plurality of encoder parameters and a plurality of classifier parameters, the neural network being trained to;

map an input data sample from the origin domain to a feature vector in a feature space in accordance with the encoder parameters; and

assign a label of the label space to the input data sample based on the feature vector in accordance with the classifier parameters;

update the encoder parameters to minimize a dissimilarity between;

a plurality of origin feature vectors computed from the origin data samples; and

a plurality of target feature vectors computed from a plurality of target data samples from a target domain, the target data samples having a smaller cardinality than the origin data samples; and

update the controller with the updated encoder parameters to control a robotic system in the target domain.

21. The non-transitory computer readable medium of claim 20, wherein the dissimilarity is computed in accordance with a sliced Wasserstein distance between the origin feature vectors in the feature space and the target feature vectors in the feature space.

22. The non-transitory computer readable medium of claim 20, wherein the instructions that cause the processor to update the encoder parameters comprise instructions that, when executed by the processor cause the processor to iteratively compute a plurality of intermediate encoder parameters, each iteration comprising:

computing the origin feature vectors in the feature space;

computing the target feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing the dissimilarity between the origin feature vectors and the target feature vectors;

updating the intermediate encoder parameters to reduce the dissimilarity between the origin feature vectors and the target feature vectors;

determining whether the dissimilarity is minimized;

in response to determining that the dissimilarity is not minimized, proceeding with another iteration with the updated intermediate encoder parameters as the intermediate encoder parameters; and

in response to determining that the dissimilarity is minimized, outputting the intermediate encoder parameters as the updated encoder parameters.

23. The non-transitory computer readable medium of claim 20, wherein the target data samples comprise a plurality of target samples and a plurality of corresponding target labels.

24. The non-transitory computer readable medium of claim 20, wherein the target data samples comprise a plurality of unlabeled target samples.

25. The non-transitory computer readable medium of claim 24, wherein the instructions that cause the processor to update the encoder parameters comprise instructions that, when executed by the processor, cause the processor to compute the updated encoder parameters by iteratively computing a plurality of intermediate encoder parameters, each iteration comprising:

computing the origin feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing the target feature vectors in the feature space in accordance with the intermediate encoder parameters;

computing predicted labels for the target feature vectors using the classifier parameters, each of the predicted labels being associated with a confidence;

defining a plurality of pseudo-labels corresponding to the predicted labels having confidences exceeding a threshold;

updating the intermediate encoder parameters based on at least one of;

minimizing a dissimilarity between the origin feature vectors and the target feature vectors; and

minimizing a classification loss of the origin data samples;

determining whether a stopping condition has been met, wherein the stopping condition comprises at least one of;

a dissimilarity between the origin feature vectors and the target feature vectors; and

a saturation of a number of the pseudo-labels between iterations;

in response to determining that the stopping condition has not been met, proceeding with another iteration with the updated intermediate encoder parameters as the intermediate encoder parameters; and

in response to determining that the stopping condition is met, outputting the intermediate encoder parameters as the updated encoder parameters.

26. The non-transitory computer readable medium of claim 25, wherein the updating the intermediate encoder parameters alternates between:

the minimizing the dissimilarity between the origin feature vectors and the target feature vectors; and

the minimizing the classification loss of the origin data samples.

27. The system of claim 20, wherein the neural network comprises a convolutional neural network, a recurrent neural network, a capsule network, or combinations thereof.