Adjusting neural networks

• US 4,912,655 A
• Filed: 12/14/1988
• Issued: 03/27/1990
• Est. Priority Date: 12/14/1988
• Status: Expired due to Fees

• Current Assignee: GTE Government Systems Corporation (General Dynamics Corporation)
• Original Assignee: GTE Laboratories Incorporated (Lumen Technologies, Inc.)
• Inventors: Wood, Laurence F.
• Primary Examiner(s): Lastova, John R.
• Application Number:
  US07/284,150
• Time in Patent Office: 468 Days
• Field of Search: 364/513, 364/800, 364/807, 364/200, 364/221.1, 364/221.3, 364/274.9, 364/276.6, 364/900, 364/916, 364/916.2, 364/917.95, 382/14, 382/15
• US Class Current: 706/25
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

A method of accelerating the training of an artificial neural network uses a computer configured as an artificial neural network with a network input and a network output, and having a plurality of interconnected units arranged in layers including an input layer and an output layer. Each unit has a multiplicity of unit inputs and a set of variables for operating upon a unit inputs to provide a unit output. The computer is programmed with a back propagation algorithm. A plurality of examples are serially provided to the network input and the network output is observed. The examples are iterated and proposed changes to each set of variables are calculated in response to feedback representing differences betwen the network output for each example and the desired output. The proposed changes are accumulated for a predetermined number of iterations, whereupon the accumulated proposed changes are added to the set of variables.

Claims

1. A method of accelerating the training of an artificial neural network comprising the steps of:

providing a computer configured as an artificial neural network with a network input and a network output, and having a plurality of interconnected units arranged in layers including an input layer and an output layer, each unit having a multiplicity of unit inputs and a set of variables for operating upon a unit inputs to provide a unit output;

programming said computer with a back propagation algorithm;

serially providing a plurality of examples to said network input and observing the network output;

iterating said examples;

calculating proposed changes to each set of variables in response to feedback representing differences between the network output for each example and the desired output;

accumulating said proposed changes for a predetermined number of iterations; and

adding the accumulated proposed changes to said set of variables after said predetermined number of iterations.