Augmenting Neural Networks with External Memory

• US 20170228637A1
• Filed: 12/30/2016
• Published: 08/10/2017
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Santoro, Adam Anthony, Wierstra, Daniel Pieter, Lillicrap, Timothy Paul, Bartunov, Sergey, Danihelka, Ivo
• Granted Patent: US 10,885,426 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 12/123 with age lists, e.g. queue, most recently used [MRU] list or least recently used [LRU] list
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/0442 characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a controller neural network that includes a Least Recently Used Access (LRUA) subsystem configured to: maintain a respective usage weight for each of a plurality of locations in the external memory, and for each of the plurality of time steps: generate a respective reading weight for each location using a read key, read data from the locations in accordance with the reading weights, generate a respective writing weight for each of the locations from a respective reading weight from a preceding time step and the respective usage weight for the location, write a write vector to the locations in accordance with the writing weights, and update the respective usage weight from the respective reading weight and the respective writing weight.

Claims

1. An augmented neural network system for processing a sequence of system inputs to generate a sequence of system outputs, the augmented neural network system comprising:

a controller neural network configured to receive a neural network input at each of a plurality of time steps and to process the neural network input to generate a neural network output for the time step, wherein each neural network output includes;

a read key, and

a write vector;

an external memory; and

a Least Recently Used Access (LRUA) subsystem that is configured to;

maintain a respective usage weight for each of a plurality of locations in the external memory that represents a strength with which the location has recently been written to or read from by the LRUA subsystem, and

for each of the plurality of time steps;

generate a respective reading weight for each of the plurality of locations in the external memory using the read key,

read data from the plurality of locations in the external memory in accordance with the reading weights,

generate a respective writing weight for each of the plurality of locations in the external memory from a respective reading weight for the location from a preceding time step and the respective usage weight for the location,

write the write vector to the plurality of locations in the external memory in accordance with the writing weights, and

update the respective usage weight for each of the plurality of locations in the external memory from the respective reading weight for the location and the respective writing weight for the location.

2. The system of claim 1, wherein the read key is different from the write vector.

3. The system of claim 1, wherein the read key is the same as the write vector.

4. The system of claim 1, wherein the LRUA subsystem is further configured to, for each of the plurality of time steps:

prior to writing the write vector to the plurality of locations and updating the respective usage weight for each of the plurality of locations;

identify a location in the external memory that has a lowest usage weight; and

erase the data stored in the identified location.

5. The system of claim 4, wherein the system further includes one or more output layers, and wherein the one or more output layers are configured to, for each of the plurality of time steps:

process the data read from the external memory for the time step to generate a system output for the time step.

6. The system of claim 1, wherein the neural network output for each of the plurality of time steps further includes a system output portion.

7. The system of claim 6, wherein the LRUA subsystem is further configured to, for each of the plurality of time steps, provide an output derived from the system output portion as a system output for the time step.

8. The system claim 7, wherein the system further includes one or more output layers, and wherein the one or more output layers are configured to, for each of the plurality of time steps:

process the data read from the external memory and the system output portion for the time step to generate a system output for the time step.

9. The system of claim 1, wherein the neural network input for each time step is a respective system input from the sequence of system inputs.

10. The system of claim 1, wherein the LRUA subsystem is further configured to, for each of the plurality of time steps:

combine the data read from the external memory and a respective system input from the sequence of system inputs to generate a subsequent neural network input for a subsequent time step.

11. The system of claim 10, wherein generating a respective writing weight for each of the plurality of locations in the external memory from a respective reading weight for the location and a preceding time step and the respective usage weight for the location comprises:

determining a convex combination of the respective reading weight for the location and the preceding time step and the respective usage weight for the location in accordance with an interpolation parameter.

12. The system of claim 1, wherein updating the respective usage weight for each of the plurality of locations in the external memory from the respective reading weight for the location and the respective writing weight for the location comprises:

applying a decay parameter to the respective usage weight for the location to determine a decayed usage weight for the location; and

determining an updated usage weight for the location by summing the decayed usage weight, the reading weight, and the writing weight for the location.

13. The system of claim 1, wherein the controller neural network has been trained using a meta-learning training technique.

14. A computer-implemented method for processing a sequence of system inputs to generate a sequence of system outputs, the method comprising:

maintaining a respective usage weight for each of a plurality of locations in an external memory that represents a strength with which the location has recently been written to or read from;

receiving a neural network input;

processing the neural network input using a controller neural network to generate a neural network output, wherein the neural network output includes a read key and a write vector;

generating a respective reading weight for each of the plurality of locations in the external memory using the read key;

reading data from the plurality of locations in the external memory in accordance with the reading weights;

generating a respective writing weight for each of the plurality of locations in the external memory from a respective reading weight for the location from a preceding time step and the respective usage weight for the location;

writing the write vector to the plurality of locations in the external memory in accordance with the writing weights; and

updating the respective usage weight for each of the plurality of locations in the external memory from the respective reading weight for the location and the respective writing weight for the location.

15. The method of claim 14, further comprising:

prior to writing the write vector to the plurality of locations and updating the respective usage weight for each of the plurality of locations;

identifying a location in the external memory that has a lowest usage weight; and

erasing the data stored in the identified location.

16. The method of claim 14, further comprising:

processing the data read from the external memory through one or more output neural network layers to generate a system output.

17. The method of claim 14, wherein the neural network input is a respective system input from the sequence of system inputs.

18. The method of claim 14, further comprising:

combining the data read from the external memory and a system input from the sequence of system inputs to generate a subsequent neural network input for a subsequent time step.

19. The method of claim 14, wherein generating a respective writing weight for each of the plurality of locations in the external memory from a respective reading weight for the location and a preceding time step and the respective usage weight for the location comprises:

determining a convex combination of the respective reading weight for the location and the preceding time step and the respective usage weight for the location in accordance with an interpolation parameter.

20. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations for generating a sequence of outputs from a sequence of inputs, the operations comprising:

maintaining a respective usage weight for each of a plurality of locations in an external memory that represents a strength with which the location has recently been written to or read from;

receiving a neural network input;

processing the neural network input using a controller neural network to generate a neural network output, wherein the neural network output includes a read key and a write vector;

generating a respective reading weight for each of the plurality of locations in the external memory using the read key;

reading data from the plurality of locations in the external memory in accordance with the reading weights;

generating a respective writing weight for each of the plurality of locations in the external memory from a respective reading weight for the location from a preceding time step and the respective usage weight for the location;

writing the write vector to the plurality of locations in the external memory in accordance with the writing weights; and

updating the respective usage weight for each of the plurality of locations in the external memory from the respective reading weight for the location and the respective writing weight for the location.