Two layer neural network comprised of neurons with improved input range and input offset

• US 5,381,515 A
• Filed: 11/05/1992
• Issued: 01/10/1995
• Est. Priority Date: 12/09/1988
• Status: Expired due to Term

• Current Assignee: Synaptics Incorporated
• Original Assignee: Synaptics Incorporated
• Inventors: Anderson, Janeen D. W., Platt, John C.
• Primary Examiner(s): Westin, Edward P.
• Assistant Examiner(s): DRISCOLL, BENJAMIN
• Application Number:
  US07/972,024
• Time in Patent Office: 796 Days
• Field of Search: 307/201, 395/24, 395/27
• US Class Current: 706/33
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: H01L 27/0629 in combination with diodes, or resistors, or capacitors
• CPC Class Codes: H03F 1/0261 with control of the polarisation voltage or current, e.g. gliding Class A
• CPC Class Codes: H03F 1/303 using a switching device H03F1/305, H03F3/005, H03F3/38 take precedence
• CPC Class Codes: H03F 3/45479 characterised by the way of common mode signal rejection
• CPC Class Codes: H03F 3/45753 using switching means, e.g. sample and hold
• CPC Class Codes: H03F 3/45977 using switching means, e.g. sample and hold

Abstract

A two-layer network according to the present invention is comprised of a first-layer array of electrically-adaptable synaptic elements, inter-layer connection circuitry comprised of electrically adaptable elements, and a second-layer array of electrically-adaptable synaptic elements. Electrons may be placed onto and removed from a floating node associated with at least one MOS transistor in each electrically adaptable element, usually comprising the gate of the transistor, in an analog manner, by application of first and second electrical control signals. A first electrical control signal controls the injection of electrons onto the floating node from an electron injection structure and the second electrical control signal controls the removal of electrons from the floating node by an electron removal structure. Each synaptic element in the synaptic array comprises an adaptable CMOS inverter or other amplifier circuit. The inputs to all first-layer synaptic elements in a row are connected to a common row input line. Adapt inputs to all synaptic elements in a column are connected together to a common column adapt line. The outputs of all first layer synaptic elements in a column are connected to a common sense amplifier on a sense line. The outputs of the sense amplifiers are connected to the inputs of the synaptic elements of the second layer of the array. The outputs of all synaptic elements in a given row in the second layer of the array are connected to a common row output line. In order to adapt the synaptic elements in the array, the voltages to which the synaptic elements in a given column of the first layer of the array is to be adapted are placed onto the input voltage lines, and the synaptic elements in column n are then simultaneously adapted by assertion of an adapt signal on the adapt line for the column. The voltages to which the synaptic elements of the second layer of the array are to be adapted are placed on the row outputs lines.

Claims

1. A two-layer synaptic array fabricated on a semiconductor substrate comprising:

a first layer comprising;

a plurality of first electrically-adaptable synaptic elements disposed in at least one row and at least one column, each of said first electrically-adaptable synaptic elements in said first layer of said array comprising an input node, an adapt-control signal node, a floating node, electron injecting means coupled to said floating node for injecting electrons on to said floating node while the voltage on said floating node is within the normal operating range of said synaptic element, negative feedback means, coupled between said floating node and said electron injecting means, and responsive to a first adapt signal on said adapt control signal node, for controlling said electron injecting means to vary the rate of injection of electrons on to said floating node in response to the magnitude of the voltage on said floating node, and an output node for supplying a current required by said synaptic element in response to a signal on said input node and said voltage on said floating node;

a row input line associated with each row of said first layer of said array, each said row input line connected to the input nodes of all of said first electrically adaptable synaptic elements associated with its row; and

a column sense line associated with each column of said first layer of said array, each column sense line connected to the output nodes of all of said first electrically adaptable synaptic elements associated with its column;

a plurality of interlayer processing elements, each of said interlayer processing elements having an input connected to one of said column-sense lines, each of said interlayer processing elements further having an output node;

a second layer comprising;

a plurality of second electrically-adaptable synaptic elements disposed in at least one row and at least one column, each of said second electrically-adaptable synaptic elements in said second layer of said array comprising an input node, an adapt-control signal node connected to, a floating node, electron injecting means coupled to said floating node for injecting electrons on to said floating node while the voltage on said floating node is within the normal operating range of said synaptic element, negative feedback means, coupled between said floating node and said electron injecting means, and responsive to a second adapt signal on said adapt-control signal node, for controlling said electron injecting means to vary the rate of injection of electrons on to said floating node in response to the magnitude of the voltage on said floating node, and an output node for supplying a current required by said synaptic element in response to a signal on said input node and said voltage on said floating node;

a second layer column input line associated with each column of said second layer of said array, each second layer column input line connected to the output node of the one of said interlayer processing elements associated with its column and to the input nodes of each of said second electrically-adaptable synaptic elements in said second layer of said array associated with its column;

a second layer row output line associated with each row in said second layer of said array, each said second layer row output line connected to the output nodes of all of said second electrically-adaptable synaptic elements in said second layer associated with its row;

a column adapt control line associated with each column of said first and second layer of said array, each said column adapt control line connected to the adapt control signal nodes of all of said first and second electrically adaptable synaptic elements associated with its column; and

means for placing adapt control signals on selected ones of said column adapt control lines to activate an adapt mode of operation of said array.

2. The synaptic array of claim 1 wherein each said electron injecting means in each of said synaptic elements in said first and second layers of said array is a semiconductor structure for performing hot electron injection.

3. The synaptic array of claim 2 wherein each of said electron injecting means is a non-avalanche hot electron injection device including:

a p-type region in said semiconductor substrate;

an n-type region disposed in said p-type region;

a floating gate disposed above said p-type region, said floating gate at least partially overlapping one edge of said n-type region and separated from the surface of said substrate by a gate oxide under a portion of said floating gate including at least where it partially overlaps the edge of said n-type region;

means for applying a first positive potential to said n-type region with respect to said p-type region to reverse bias said n-type region, said positive potential having a magnitude greater than about 2 volts relative to said p-type region, but less than the voltage required to induce avalanche breakdown between said n-type region and said p-type region;

means for capacitively coupling a second positive potential to said floating gate, said second positive potential having a magnitude of greater than about 2 volts relative to said p-type region;

means for injecting electrons into said p-type region;

whereby said first and second positive potentials act to accelerate said electrons to an energy sufficient to surmount the barrier energy of said gate oxide and thereby inject said electrons onto said floating gate.

4. The synaptic array of claim 1 wherein each of said interlayer processing elements comprises an electrically-adaptable element including an input connected to one of said column sense lines of said first layer, an output connected to one of said input lines of said second layer, a floating node coupled to said input, electron injecting means coupled to said floating node for injecting electrons on to said floating node while the voltage on said floating node is within the normal operating range of said synaptic element, negative feedback means, coupled between said floating node and said electron injecting means, and responsive to a signal on an adapt-control signal node, for controlling said electron injecting means to vary the rate of injection of electrons on to said floating node in response to the magnitude of the voltage on said floating node, and further including means for selectively placing an adapt control signal on said adapt control signal node of said interlayer processing element.

5. The synaptic array of claim 1 wherein each of said synaptic elements in said first and second layers of said array further includes electron removal means coupled to said floating node for removing electrons from said floating node while the voltage on said floating node is within the normal operating range of said synaptic element.

6. The synaptic array of claim 5 wherein said electron removal means is a conductor-insulator-conductor structure for performing electron tunneling.

7. The synaptic array of claim 6 wherein said floating node is a layer of polysilicon and said electron removal means includes a second layer of polysilicon separated from said floating node by a layer of SiO₂.

8. The synaptic array of claim 5, wherein each synaptic element is further comprised of a negative feedback means, coupled between said floating node and said electron removal means, and responsive to adapt control signals on its respective adapt control signal node for controlling said electron removal means to vary the rate of removal of electrons from said floating node in response to the magnitude of the voltage on said floating node.

9. The synaptic array of claim 1, wherein said first layer of said array further includes a sample/hold circuit connected between the input of each of said first electrically-adaptable synaptic element and its associated row input line, each of said sample/hold circuits including a control input, the control inputs of all said sample/hold circuits associated with each column of said first layer of said array connected to a column sample control line.

10. The synaptic array of claim 1 wherein each of said synaptic elements in said first layer of said array are comprised of electrically-adaptable inverters.

11. The synaptic array of claim 1 wherein each of said synaptic elements in said second layer of said array are blending synapses.

12. A two-layer synaptic array fabricated on a semiconductor substrate comprising:

a first layer comprising;

a plurality of first electrically-adaptable synaptic elements disposed in at least one row and at least one column, each of said first electrically-adaptable synaptic elements comprising an input node, an output node, a current sense node, an inverter including a P-Channel MOS transistor having a drain, a source connected to said current sense node, and a gate connected to a floating node, and an N-Channel MOS transistor having a drain connected to the drain of said P-Channel MOS transistor and to said output node, a gate connected to said floating node, and a source connected to a source of a fixed voltage potential, a capacitor connected between said input node and said floating node, an adapt control signal node, electron injecting means coupled to said floating node for injecting electrons on to said floating node while the voltage on said floating node is within the normal operating range of said inverter, injection control means, responsive to said adapt signal and the voltage on said output node, for generating an electrical injection control signal to control said electron injecting means to increase the rate of injection of electrons on to said floating node in response to an increase in voltage on said floating node and to decrease the rate of injection of electrons on to said floating node in response to a decrease in voltage on said floating node, said current sense node for supplying a current required by said synaptic element at the source of said P-Channel MOS transistor in response to a signal on said input node and said voltage on said floating node;

a row input line associated with each row of said first layer of said array, each said row input line connected to the input nodes of all of said electrically adaptable synaptic elements associated with its row; and

a column sense line associated with each column of said first layer of said array, each column sense line connected to the current sense nodes of all of said electrically adaptable synaptic elements associated with its column;

a plurality of interlayer processing elements, individual ones of said interlayer processing elements having an input connected to one of said column-sense lines, and an output node;

a second layer comprising;

a plurality of second electrically-adaptable synaptic elements disposed in at least one row and at least one column, said at least one column corresponding to said at least one column of synaptic elements in said first layer, each of said second electrically-adaptable synaptic elements in said second layer of said array comprising an input node, an output node, a current sense node, a switching element connected between said output node and said current sense node, said switching element having a control element, a transconductance amplifier having a bias input connected to said input node, an inverting input connected to a floating node, a non-inverting input connected to a fixed voltage source, and an output connected to said output node, said current sense node for supplying a current required by said synaptic element in response to a signal on said input node and a voltage on said floating node, a first capacitor having a first plate connected to a fixed voltage source and a second plate connected to said floating node, a second capacitor having a first plate connected to said floating node and a second plate connected to said current sense node, an adapt control signal node, electron injecting means coupled to said floating node for injecting electrons on to said floating node while the voltage on said floating node is within the normal operating range of said amplifier, negative feedback means, coupled between said floating node and said electron injection means, and responsive to said signal on said adapt control signal node for controlling said electron injecting means to vary the rate of injection of electrons on to said floating node in response to the magnitude of the voltage on said floating node;

a second layer column input line associated with each column in said second layer of said array, each second layer column input line connected to the output node of the one of said interlayer processing elements associated with its column and to the inputs of each of said electrically-adaptable synaptic elements in said second layer of said array associated with its column;

a second layer row output line associated with each row in said second layer of said array, each said second layer row output line connected to the current sense nodes of all of said electrically adaptable synaptic elements in said second layer associated with its row;

a column adapt control input line associated with each corresponding column of said first and second layers of said array, each said column adapt control input line connected to the adapt control signal nodes of all of said electrically adaptable synaptic elements associated with its column in both said first and second layers of said array;

means for placing an adapt control input signal on a selected one of said column adapt control input lines and to said control element of each of said switching elements in said second electrically-adaptable synaptic elements so as to disconnect said output node from said current sense node to activate an adapt mode of operation of said array;

means for placing a desired vector of input voltages on the row input lines of said first layer of the array during said adapt mode of operation; and

means for placing a desired vector of output voltages on the row sense lines of said second layer of the array during said adapt mode of operation.

13. The synaptic array of claim 12 wherein each said electron injecting means in each of said synaptic elements in said first and second layers of said array is a semiconductor structure for performing hot electron injection.

14. The synaptic array of claim 13 wherein each of said electron injecting means is a non-avalanche hot electron injection device including:

a p-type region in said semiconductor substrate;

an n-type region disposed in said p-type region;

a floating gate disposed above said p-type region, said floating gate at least partially overlapping one edge of said n-type region and separated from the surface of said substrate by a gate oxide under a portion of said floating gate including at least where it partially overlaps the edge of said n-type region;

means for applying a first positive potential to said n-type region with respect to said p-type region to reverse bias said n-type region, said positive potential having a magnitude greater than about 2 volts relative to said p-type region, but less than the voltage required to induce avalanche breakdown between said n-type region and said p-type region;

means for capacitively coupling a second positive potential to said floating gate, said second positive potential having a magnitude of greater than about 2 volts relative to said p-type region;

means for injecting electrons into said p-type region;

whereby said first and second positive potentials act to accelerate said electrons to an energy sufficient to surmount the barrier energy of said gate oxide and thereby inject said electrons onto said floating gate.

15. The synaptic array of claim 12 wherein each of said interlayer processing element comprises an electrically-adaptable element including a floating node, electron injecting means coupled to said floating node for injecting electrons on to said floating node while the voltage on said floating node is within the normal operating range of said electrically adaptable element, negative feedback means, coupled between said floating node thereof, and said electron injection means, and responsive to a signal on an adapt-control signal node, for controlling said electron injection means to vary the rate of injection of electrons on to said floating node in response to the magnitude of the voltage on said floating node, and further including means for selectively placing an adapt control signal on said adapt control signal node of said interlayer processing element.

16. The synaptic array of claim 12 wherein each of said synaptic elements in said first and second layers of said array further includes electron removal means coupled to said floating node for removing electrons from said floating node while the voltage on said floating node is within the normal operating range of said synaptic element.

17. The synaptic array of claim 16 wherein said electron removal means is a conductor-insulator-conductor structure for performing electron tunneling.

18. The synaptic array of claim 17 wherein said floating node is a layer of polysilicon and said electron removal means includes a second layer of polysilicon separated from said floating node by a layer of SiO₂.

19. The synaptic array of claim 16, wherein each synaptic element is further comprised of a negative feedback means, coupled between said floating node and said electron removal means, and responsive to adapt control signals on its respective adapt control signal node for controlling said electron removal means to vary the rate of removal of electrons from said floating node in response to the magnitude of the voltage on said floating node.

20. The synaptic array of claim 12 wherein each of said synaptic elements in said first layer of said array are comprised of electrically-adaptable inverters.

21. The synaptic array of claim 12 wherein each of said synaptic elements in said second layer of said array are interpolating synapses.

22. The synaptic array of claim 12, wherein said array further includes:

a plurality of first sample/hold circuits, one of said first sample/hold circuits connected between the input of each of said first electrically-adaptable synaptic element and its associated row input line, each of said first sample/hold circuits including a control input;

a plurality of second sample/hold circuits, one of said second sample/hold circuits connected between the input of each of said second electrically-adaptable synaptic element and its associated column input line, each of said second sample/hold circuits including a control input;

a sample control line associated with each column of said array, each sample control line connected to the control inputs of the ones of said first and second sample/hold circuits associated with its column; and

sample control means, coupled to said sample control lines, for selectively activating said sample control lines during an adapt mode of said synaptic array.