MIRROR LOSS NEURAL NETWORKS

• US 20190314985A1
• Filed: 03/19/2018
• Published: 10/17/2019
• Est. Priority Date: 03/17/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Sermanet, Pierre
• Granted Patent: US 11,453,121 B2
• Time in Patent Office: Days
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06V 10/70 using pattern recognition or machine learning optical pattern recognition or electronic computations therefor G06V10/88
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

This description relates to a neural network that has multiple network parameters and is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment. The neural network can be used to control a robotic agent. The network can be trained using a method comprising: obtaining a first observation captured by a first modality; obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality; obtaining a third observation captured by the first modality that is not co-occurring with the first observation; determining a gradient of a triplet loss that uses the first observation, the second observation, and the third observation; and updating current values of the network parameters using the gradient of the triplet loss.

Claims

1. A method of training a neural network having a plurality of network parameters, wherein the neural network is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment, the method comprising:

obtaining a first observation captured by a first modality;

obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality;

obtaining a third observation captured by the first modality that is not co-occurring with the first observation;

determining a gradient of a triplet loss that uses the first observation as an anchor example, the second observation as a positive example, and the third observation as a negative example; and

updating current values of the network parameters using the gradient of the triplet loss.

2. The method of claim 1, wherein the observations are images, wherein the first modality is a camera at a first viewpoint, and wherein the second modality is a camera at a second, different viewpoint.

3. The method of claim 1, wherein the third observation is within a temporal neighborhood of the first observation.

4. The method of claim 3, wherein obtaining the third observation comprises:

selecting the third observation randomly from the observations captured by the first modality that are within the temporal neighborhood of the first observation.

5. The method of claim 3, wherein obtaining the third observation comprises:

selecting an observation that is a hard negative relative to the first observation from the observations captured by the first modality that are within the temporal neighborhood of the first observation.

6. The method of claim 1, wherein determining the gradient of the triplet loss comprises:

processing the first observation using the neural network in accordance with the current values of the network parameters to generate a first embedding;

processing the second observation using the neural network in accordance with the current values of the network parameters to generate a second embedding;

processing the third observation using the neural network in accordance with the current values of the network parameters to generate a third embedding; and

determining the triplet loss from (i) a first distance between the first embedding and the second embedding and (ii) a second distance between the first embedding and the third embedding.

7. The method of claim 1, wherein the neural network is further configured to process the numeric embedding to generate a control output defining a control input to an agent interacting with the environment.

8. The method of claim 7, wherein training the neural network further comprises training the neural network to generate control outputs that cause the agent to imitate another agent interacting with the environment.

9. The method of claim 1, wherein the first modality is a camera that captures a self-view of the agent interacting with the environment, and wherein training the neural network further comprises training the neural network to generate control outputs that position joints of the agent in a same position as in received captured self-views.

10. The method of claim 7, wherein the agent is a robotic agent.

11. The method of claim 1, wherein the environment includes an agent performing a specified task.

12. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers, cause the one or more computers to perform operations for training a neural network having a plurality of network parameters, wherein the neural network is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment, the operations comprising:

obtaining a first observation captured by a first modality;

obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality;

obtaining a third observation captured by the first modality that is not co-occurring with the first observation;

determining a gradient of a triplet loss that uses the first observation as an anchor example, the second observation as a positive example, and the third observation as a negative example; and

updating current values of the network parameters using the gradient of the triplet loss.

13. One or more non-transitory computer storage media storing instructions that when executed by one or more computers, cause the one or more computers to perform operations for training a neural network having a plurality of network parameters, wherein the neural network is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment, the operations comprising:

obtaining a first observation captured by a first modality;

obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality;

obtaining a third observation captured by the first modality that is not co-occurring with the first observation;

determining a gradient of a triplet loss that uses the first observation as an anchor example, the second observation as a positive example, and the third observation as a negative example; and

updating current values of the network parameters using the gradient of the triplet loss.

18. The one or more non-transitory computer storage media of claim 13, wherein the observations are images, wherein the first modality is a camera at a first viewpoint, and wherein the second modality is a camera at a second, different viewpoint.

19. The one or more non-transitory computer storage media of claim 13, wherein the third observation is within a temporal neighborhood of the first observation.

20. The one or more non-transitory computer storage media of claim 19, wherein obtaining the third observation comprises:

selecting the third observation randomly from the observations captured by the first modality that are within the temporal neighborhood of the first observation.

21. The one or more non-transitory computer storage media of claim 19, wherein obtaining the third observation comprises:

selecting an observation that is a hard negative relative to the first observation from the observations captured by the first modality that are within the temporal neighborhood of the first observation.

22. The one or more non-transitory computer storage media of claim 13, wherein determining the gradient of the triplet loss comprises:

processing the first observation using the neural network in accordance with the current values of the network parameters to generate a first embedding;

processing the second observation using the neural network in accordance with the current values of the network parameters to generate a second embedding;

processing the third observation using the neural network in accordance with the current values of the network parameters to generate a third embedding; and

determining the triplet loss from (i) a first distance between the first embedding and the second embedding and (ii) a second distance between the first embedding and the third embedding.

23. The one or more non-transitory computer storage media of claim 13, wherein the neural network is further configured to process the numeric embedding to generate a control output defining a control input to an agent interacting with the environment.

24. The one or more non-transitory computer storage media of claim 23, wherein training the neural network further comprises training the neural network to generate control outputs that cause the agent to imitate another agent interacting with the environment.

14-17. -17. (canceled)