Neural semiconductor chip and neural networks incorporated therein

• US 5,717,832 A
• Filed: 06/07/1995
• Issued: 02/10/1998
• Est. Priority Date: 07/28/1994
• Status: Expired due to Term

• Current Assignee: Associative Computing, Inc., in2H2, Inc., Silicon Recognition Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Steimle, Andre, Paillet, Guy, Tannhof, Pascal
• Primary Examiner(s): Hafiz, Tariq R.
• Application Number:
  US08/488,443
• Time in Patent Office: 979 Days
• Field of Search: 395/20-25, 395/27, 395/11, 382/155-159
• US Class Current: 706/33
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/955 using specific electronic processors

Abstract

A base neural semiconductor chip (10) including a neural network or unit (11(#)). The neural network (11(#)) has a plurality of neuron circuits fed by different buses transporting data such as the input vector data, set-up parameters, and control signals. Each neuron circuit (11) includes logic for generating local result signals of the "fire" type (F) and a local output signal (NOUT) of the distance or category type on respective buses (NR-BUS, NOUT-BUS). An OR circuit (12) performs an OR function for all corresponding local result and output signals to generate respective first global result (R*) and output (OUT*) signals on respective buses (R*-BUS, OUT*-BUS) that are merged in an on-chip common communication bus (COM*-BUS) shared by all neuron circuits of the chip. In a multi-chip network, an additional OR function is performed between all corresponding first global result and output signals (which are intermediate signals) to generate second global result (R**) and output (OUT**) signals, preferably by dotting onto an off-chip common communication bus (COM**-BUS) in the chip'"'"'s driver block (19). This latter bus is shared by all the base neural network chips that are connected to it in order to incorporate a neural network of the desired size. In the chip, a multiplexer (21) may select either the intermediate output or the global output signal to be fed back to all neuron circuits of the neural network, depending on whether the chip is used in a single or multi-chip environment via a feed-back bus (OR-BUS). The feedback signal is the result of a collective processing of all the local output signals.

Claims

1. A base neural network (11(A)) for identifying an input vector (A), said base neural network (11(A)) comprising:

a neural network (11-(#)) comprised of a plurality of neuron circuits (11-i) being connected in parallel to a data bus and a control bus, each said neuron circuit being in one of two states, said two states being a free state and an engaged state, each said neuron circuit comprising;

means for generating local result signals, said local result signals being individual responses of said neuron circuit to the input vector,

means for placing said generated local result signals on a first dedicated bus (NR-BUS) after said neuron is engaged;

means for generating a local output signal (NOUT), said local output signal indicating a vector category or a distance between said input vector and a prototype vector stored in said neuron circuit, and

means for placing said local output signal (NOUT) on a second dedicated bus (NOUT-BUS) after said neuron is engaged;

logic means (12) for combining said local result signals and generating therefrom corresponding global result signals, and for combining said local output signals and generating therefrom a global output signal (OUT*), said global result signals and said global output signals being on their respective buses (R*-BUS, OUT*-BUS) and merging in a common communication bus (COM*-BUS); and

,

means for feeding back at least said global output signal as a feed-back signal (OR) to each of said neuron circuits on a feed-back bus (OR-BUS), thereby allowing the comparison between said local output and said global output signal to generate a local result signal.

2. The base neural network of claim 1 wherein each said neuron circuit further comprises:

means for storing a local category (Ci) when said neuron circuit is engaged, said local category (Ci) being a value different than zero;

means for storing a prototype vector (B) as said neuron circuit is engaged;

means for calculating a distance (Di) between said input vector (A) and said prototype vector (B) after said neuron circuit is engaged; and

,

means for storing an Actual Influence Field value when said neuron circuit is engaged.

3. The base neural network of claim 2 wherein said local result signals for each engaged said neuron circuit comprises:

a fire signal (F), said fire signal being active if said neuron circuit recognizes the input vector (A);

a degenerate signal (DEGOUT), said degenerate signal being active if said fire signal is active and said engaged neuron circuit has degenerated; and

,

an ambiguity signal (UNC/FIRE.OK), said ambiguity signal being driven active if said fire signal is active and at least one other neuron fire signal is active and if said engaged neuron circuit local category does not match the local category of said other firing neuron.

4. The base neural network of claim 3 wherein said local result signals are OR'"'"'ed with corresponding local result signals from each of said plurality of neuron circuits to generate said global result signals.

5. The base neural network of claim 3 wherein each said local ambiguity signal (UNC/FIRE.OK) results from comparing a local category (Ci) with said global output signal said global output signal (C*).

6. The base neural network of claim 3 further comprising an additional global result signal (ID*) resulting from ANDing the global fire signal (F*) and the inverse of the global ambiguity signal (UNC/FIRE.OK*), whereby ID*=F, AND UNC/FIRE.OK*.

7. The base neural network of claim 3 wherein said global output signal (OUT*) is the smallest distance calculated (Dmin) by the engaged neuron circuits of the neural network, whereby OUT*=Dmin.

8. The base neural network of claim 2 wherein said local output signals are OR'"'"'ed with corresponding local output signals from each of said plurality of neuron circuits to generate said global output signal (OUT*).

9. The base neural network of claim 1 wherein said logic means for combining is an OR circuit (12).

10. The base neural network of claim 9 wherein said OR circuit consists of a plurality of OR sub-circuits.

11. The base neural network of claim 10 wherein each local result signal is combined in one of said OR sub-circuits (12.2 to 12.4) and the local output signals from each neuron circuit are combined in OR sub-circuit (12.1).

12. The base neural network of claim 1 wherein said logic means for combining consists of dot ORing.

13. The base neural network of claim 12 wherein a plurality of open drain devices are dot ORed together.

14. The base neural network of claim 1 further including selection means for selecting only engaged neuron circuits in the neural network.

15. The base neural network of claim 14 further including second selection means for selecting only a determined number among the engaged neuron circuits.

16. A base neural network (11(A)) for identifying an input vector (A), said base neural network comprising:

a neural network (11-(#)) comprised of a plurality of neuron circuits (11-i) connected in parallel to a data bus and a control bus, each said neuron circuit being in one of two states, said two states being a free state and an engaged state, each said neuron circuit comprising;

means for generating local result signals, said local results being individual responses of said neuron circuit to the input vector,

means for placing said generated local results on a first dedicated bus (NR-BUS) after said neuron circuit is engaged,

means for generating a local output signal (NOUT), and

means for placing said local output signal (NOUT) on a second dedicated bus (NOUT-BUS) after said neuron circuit is engaged;

first logic means (12) for combining a group of said local result signals to generate corresponding intermediate result signals and for combining a group of said local output signals to an generate intermediate output signal (OUT*), said intermediate signals being available on their respective buses (R*-BUS, OUT*-BUS) and merging in a first common communication bus (COM*-BUS);

second logic means for combining at least two of each of said intermediate result signals to generate corresponding second global result signals and for combining at least two of said intermediate output signals to generate a global output signal (OUT**), said global result signal and said global output signal being available on their respective buses (R**-BUS, OUT**-BUS) and merging in a second common communication bus (COM**-BUS); and

,

means for feeding back at least said global output signal to each of said neuron circuits as a feed-back signal (OR) on a feed-back bus (OR-BUS), thereby allowing the comparison between corresponding said local output signal and said global output signal to generate a local result signal.

17. The base neural network of claim 16 wherein each said neuron circuit further includes:

means for storing a local category (Ci) when the neuron circuit is engaged, said stored local category being different from zero;

means for storing a prototype vector (B) as said neuron circuit is engaged;

means for calculating the distance (Di) between said input vector (A) and the prototype vector (B) when said neuron circuit is engaged; and

,

means for storing an Actual Influence Field (AIF) valve when said neuron circuit is engaged.

18. The base neural network of claim 17 wherein said local result signals for each said engaged neuron circuit comprise:

a fire signal (F), said fire signal being driven active if said neuron circuit recognizes the input vector;

a degenerate signal (DEGOUT), said degenerate signal being driven active if said fire signal is active and said neuron circuit has degenerated; and

,

an ambiguity signal (UNC/FIRE.OK) said ambiguity signal being driven active if said fire signal is active and at least one other neuron fire signal is active and if said engaged neuron circuit local category does not match the local category of said other firing neuron.

19. The base neural network of claim 18 wherein each of said second global result signals is the result of ORing of all corresponding local result signals.

20. The base neural network of claim 18 wherein when said second global output signal (OUT**) represents a distance, the distance represented is the smallest distance calculated (Dmin) by the engaged neuron circuits of the neural network, whereby OUT**=Dmin.

21. The base neural network of claim 17 wherein said ambiguity signal (UNC/FIRE.OK) is the result of comparing between the local category (Ci) and said second global output signal (C*).

22. The base neural network of claim 17 further including a second global result signal (ID**) resulting from ANDing the global result fire signal (F**) and the inverse of the global result ambiguity signal (UNC/FIRE.OK**), whereby F** AND UNC/FIRE.OK.

23. The base neural network of claim 16 wherein when said second global output signal (OUT**) is a global category and is the result of ORing all of the local categories (OUT**=C*=C1 OR . . . OR CN).

24. The base neural network of claim 16 wherein said first logic means for combining consists of an OR circuit (12).

25. The base neural network of claim 24 wherein said OR circuit consists of a plurality of OR sub-circuits.

26. The base neural network of claim 25 wherein each local result signal is combined in one of said OR sub-circuits (12.2 to 12.4) and the local output signals from each neuron circuit are combined in one OR sub-circuit (12.1).

27. The base neural network of claim 16 wherein said second logic means for combining consists of a dot logic function.

28. The base neural network of claim 27 wherein open drain devices are used for said dot logic function.

29. The base neural network of claim 16 further including selection means for selecting only engaged neuron circuits in the neural network.

30. The base neural network of claim 29 further including additional selection means for selecting only a determined number among the engaged neuron circuits.

40. A neural module formed by the assembly of at least two base neural chips in claims 16, 31 or 33.

31. A base neural semiconductor chip (10) comprising:

a neural network (11(#)) comprised of a plurality of neuron circuits (11-i) connected in parallel to a data and a control bus, each said neuron circuit being in one of two states, said two states being a free state and an engaged state, each said neuron circuit comprising;

means for generating local result signals said local result signals being individual responses of said neuron circuit to an input vector,

means for placing said generated local result signals on a first dedicated bus (NR-BUS), after said neuron is engaged, and

means for generating a local output signal (NOUT), said local output signal indicating a vector category or a distance between said input vector and a prototype vector stored in said neuron circuit, and

means for placing said local output on a second dedicated bus (NOUT-BUS) after said neuron is engaged;

logic means for combining said local result signals and generating therefrom corresponding global result signals and, for combining said local output signals and generating therefrom global output signals (OUT*) said global result signals and said global output signals being on their respective buses (R*-BUS, OUT*-BUS) and merging in a common communication bus (COM*-BUS); and

means for feeding back at least said global output signal as a feedback signal (OR) to each of said neuron circuits on a feedback bus (OR-BUS), thereby allowing the comparison between corresponding said local output signals and said global output signal to generate said local result signals.

32. The base neural semiconductor chip of claim 31 further comprising an interconnecting communication bus.

33. A base neural semiconductor chip (10) for use as a unit in a multi-chip network, said base neural semiconductor chip comprising:

a neuron unit (11(#)) comprised of a plurality of neuron circuits (11-i) connected in parallel to a data bus and a control bus, each said neuron circuit being in one of two states, said two states being a free state and an engaged state, each said neuron circuit comprising;

means for generating local result signals said local result signals being individual responses of said neuron circuit to an input vector,

means for placing said generated local result signals on a first dedicated bus (NR-BUS) after said neuron is engaged, and

means for generating a local output signal (NOUT), said local output signal indicating a vector category or a distance between said input vector and a prototype vector stored in said neuron circuit, and

means for placing said local output signal on a second dedicated bus (NOUT-BUS);

first logic means for combining a group of said local result signals and generating therefrom corresponding first intermediate result signals and for combining a group of first local output signals generating therefrom a first intermediate output signal (OUT*), said intermediate result signals and said intermediate output signals being available on their respective buses (R*-BUS, OUT*-BUS) and merging in a first common communication bus (COM*-BUS);

second logic means for combining at least two of each of said intermediate result signals to generate corresponding second global result signals and for combining at least two of said intermediate output signals to generate a global output signal (OUT**) on their respective buses (R**-BUS, OUT**-BUS) of a second common communication bus (COM**-BUS), said second common communication bus being commonly connectable to other base neural semiconductor chips in a network of said chips; and

,

means for feeding back at least said global output signal as a feedback signal (OR) to each of said neuron circuits on a feedback bus (OR-BUS) thereby allowing the comparison between corresponding said local output signal and said global output signal to generate said local result signals.

34. The base neural semiconductor chip of claim 33 further including multiplexer means to selectively feed back either the intermediate output signal or the global output signal.

35. The base neural semiconductor of claim 33 or 34 wherein each said neuron circuit further includes:

means for storing the local category (Ci) when said neuron circuit is engaged, said local category being different from zero;

means for storing a prototype vector (B) as said neuron circuit is engaged;

means for calculating the distance (Di) between an input vector (A) and a prototype vector (B) after said neuron circuit is engaged; and

,

means for storing the Actual Influence Field (AIF) when said neuron circuit is engaged.

36. The base neural semiconductor chip of claim 35 wherein said first logic means consists of an OR circuit (12).

37. The base neural semiconductor chip of claim 35 wherein said first logic means consists of a dot logic function.

38. The base neural semiconductor chip of claim 37 further including selection means for selecting only engaged neuron circuits in the neural network.

39. The base neural semiconductor network of claim 38 further including additional selection means for selecting only a determined number among said selected engaged neuron circuits.