Digital processing element in an artificial neural network

• US 5,216,751 A
• Filed: 06/12/1992
• Issued: 06/01/1993
• Est. Priority Date: 10/22/1990
• Status: Expired due to Fees

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Peterson, William M., Gardner, Robert M., Leivian, Robert H., Garrison, Sidney C. III
• Primary Examiner(s): Fleming, Michael R.
• Assistant Examiner(s): Downs, Robert W.
• Application Number:
  US07/898,189
• Time in Patent Office: 354 Days
• Field of Search: 395/27, 395/24
• US Class Current: 706/42
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

An artificial neural network is provided using a digital architecture having feedforward and feedback processors interconnected with a digital computation ring or data bus to handle complex neural feedback arrangements. The feedforward processor receives a sequence of digital input signals and multiplies each by a weight in a predetermined manner and stores the results in an accumulator. The accumulated values may be shifted around the computation ring and read from a tap point thereof, or reprocessed through the feedback processor with predetermined scaling factors and combined with the feedforward outcomes for providing various types neural network feedback computations. Alternately, the feedforward outcomes may be placed sequentially on a data bus for feedback processing through the network. The digital architecture includes a predetermined number of data input terminals for the digital input signal irrespective of the number of synapses per neuron and the number of neurons per neural network, and allows the synapses to share a common multiplier and thereby reduce the physical area of the neural network. A learning circuit may be utilized in the feedforward processor for real-time updating the weights thereof to reflect changes in the environment.

Claims

1. A neural network, comprising:

a first digital input bus coupled for receiving a first digital input signal;

a digital address bus coupled for receiving a digital address signal;

first means for storing a plurality of digital weighting elements, said first means being coupled to said digital address bus for addressing one of said plurality of digital weighting elements according to said digital address signal and loading said one of said plurality of digital weighting elements onto an output port, said first means including an input port for receiving data for storage as one of said plurality of digital weighting elements;

second means coupled for receiving said first digital input signal and said one of said plurality of digital weighting elements for providing an output signal as the product of said first digital input signal and said one of said plurality of digital weighting elements;

third means coupled for receiving said output signal of said second means and accumulating the value thereof; and

fourth means responsive to said plurality of digital weighting elements and second and third digital input signals for altering the values of said plurality of digital weighting elements stored in said first means, said fourth means including,

(a) a subtracting circuit having first and second inputs and having an output, said first input being coupled to said output port of said first means for receiving ones of said plurality of digital weighting elements, said second input being coupled for receiving said second digital input signal representative of a previous series of said first digital input signals, said output providing a difference signal from said one of said plurality of digital weighting elements and said second digital input signal,

(b) a first multiplication circuit having first and second inputs and having an output, said first input being coupled to said output of said subtracting circuit, said second input being coupled for receiving a third digital input signal for weighting said difference signal, and

(c) a summing circuit having first and second inputs and having an output, said first input being coupled to said output of said first multiplication circuit, said second input being coupled to said output port of said first means, said output being coupled to said input port of said first means.

2. The neural network of claim 1 wherein said digital address signal and said first digital input signal change value such that ones of said plurality of digital weighting elements are respectively multiplied by said first digital input signal for providing a plurality of output signals of said second means over time which are accumulated in said third means.

3. The neural network of claim 2 wherein said first means includes a plurality of digital memory locations addressable by said digital address signal and having a common output coupled to said output port of said first means.

4. The neural network of claim 3 wherein said second means includes a second multiplication circuit having first and second inputs and an output, said first input being coupled to said digital input bus, said second input being coupled to said output port of said first means for receiving said one of said plurality of digital weighting elements, said output providing said output signal of said second means as the product of said first digital input signal and said one of said plurality of digital weighting elements.

5. The neural network of claim 4 wherein said third means includes a digital accumulating circuit having an input coupled to said output of said second multiplication circuit and having an output for providing an output signal as the sum total of said plurality of output signals of said second means.

6. In a neural network a plurality of digital processing elements, each of said plurality of digital processing elements being coupled to a first digital input bus for receiving a first digital input signal and to a digital address bus for receiving a digital address signal and providing an output signal at an output, one of said plurality of digital processing elements comprising:

first means for storing a plurality of digital weighting elements, said first means being coupled to the digital address bus for addressing one of said plurality of digital weighting elements according to the digital address signal and loading said one of said plurality of digital weighting elements onto an output port;

second means coupled for receiving the first digital input signal and said one of said plurality of digital weighting elements for providing an output signal as the product of the first digital input signal and said one of said plurality of digital weighting elements;

third means coupled for receiving said output signal of said second means and providing an output signal at an output as the accumulation thereof; and

fourth means responsive to said output signal of said third means and the output signals of ones of said plurality of digital processing elements for accumulating the values thereof and providing said output signal of said one of said plurality of processing elements, said fourth means including,

(a) a first multiplexer circuit responsive to first and second states of a first control signal for selecting between said output signal of said third means applied at a first input and a digital signal applied at a second input, respectively, for providing an output signal,

(b) a register circuit having an input coupled for receiving said output signal of said first multiplexer circuit and having an output,

(c) a first multiplier circuit having first and second inputs and an output, said first input being coupled for receiving a second digital input signal, said second input being coupled for receiving one of the output signals of said plurality of digital processing elements, said output providing an output signal as the product thereof, and

(d) an adder circuit having first and second inputs and an output, said first input being coupled to said output of said register circuit, said second input being coupled for receiving said output signal of said first multiplier circuit, said output providing said digital signal applied at said second input of said first multiplexer circuit as the summation thereof.