Neural network apparatus and method for pattern recognition

• US 5,058,180 A
• Filed: 04/30/1990
• Issued: 10/15/1991
• Est. Priority Date: 04/30/1990
• Status: Expired due to Term

• Current Assignee: National Semiconductor Corporation (Texas Instruments, Inc.)
• Original Assignee: National Semiconductor Corporation (Texas Instruments, Inc.)
• Inventors: Khan, Emdadur R.
• Primary Examiner(s): NOT, DEFINED
• Assistant Examiner(s): NOT, DEFINED
• Application Number:
  US07/516,753
• Time in Patent Office: 533 Days
• Field of Search: 382/14, 382/15, 382/36, 364/274.9, 364/972.4
• US Class Current: 382/158
• CPC Class Codes: G06F 18/23211 with adaptive number of clusters
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06V 30/19107 Clustering techniques

Abstract

A self-organizing neural network having input and output neurons mutually coupled via bottom-up and top-down adaptive weight matrics performs pattern recognition while using substantially fewer neurons and being substantially immune from pattern distortion or rotation. The network is first trained in accordance with the adaptive resonance theory by inputting reference pattern data into the input neurons for clustering within the output neurons. The input neurons then receive subject pattern data which are transferred via a bottom-up adaptive weight matrix to a set of output neurons. Vigilance testing is performed and multiple computed vigilance parameters are generated. A predetermined, but selectively variable, reference vigilance parameter is compared individually against each computed vigilance parameter and adjusted with each comparison until each computed vigilance parameter equals or exceeds the adjusted reference vigilance parameter, thereby producing an adjusted reference vigilance parameter for each output neuron. The input pattern is classified according to the output neuron corresponding to the maximum adjusted reference vigilance parameter. Alternatively, the original computed vigilance parameters can be used by classifying the input pattern according to the output neuron corresponding to the maximum computer vigilance parameter.

Claims

1. An improved self-organizing neural network coupled to perform adaptive pattern recognition in accordance with adaptive resonance theory, said neural network comprising a plurality of input neurons coupled to accept input pattern data representing a pattern, a plurality of output neurons, a bottom-up adaptive weight matrix coupling said input neurons to said output neurons, a top-down adaptive weight matrix coupling said output neurons to said input neurons, and vigilance parameter computing means for computing a plurality of vigilance parameters based substantially upon said input pattern data, said improvement comprising:

memory means for storing said plurality of computed vigilance parameters; and

pattern classification means for classifying said pattern according to the maximum computed vigilance parameter among said plurality of computed vigilance parameters.

2. An improved self-organizing neural network coupled to perform adaptive pattern recognition in accordance with adaptive resonance theory, said neural network comprising a plurality of input neurons coupled to accept input pattern data representing a pattern, a plurality of output neurons, a bottom-up adaptive weight matrix coupling said input neurons to said output neurons, a top-down adaptive weight matrix coupling said output neurons to said input neurons, vigilance parameter computing means for computing a plurality of vigilance parameters based substantially upon said input pattern data, and vigilance parameter comparator means for individually comparing said plurality of computed vigilance parameters with a predetermined reference vigilance parameter, said improvement comprising:

vigilance parameter adjustment means for selectively adjusting said predetermined reference vigilance parameter in response to said comparisons thereof with said plurality of computed vigilance parameters, said adjustment resulting in a plurality of adjusted reference vigilance parameters;

memory means for storing said plurality of adjusted reference vigilance parameters; and

pattern classification means for classifying said pattern according to the maximum adjusted reference vigilance parameter among said plurality of adjusted reference vigilance parameters.

3. A neural network coupled to perform adaptive pattern recognition, said neural network comprising:

vigilance parameter computing means for computing a plurality of vigilance parameters based substantially upon input pattern data representing a pattern;

memory means for storing said plurality of computed vigilance parameters; and

pattern classification means for classifying said pattern according to the maximum computed vigilance parameter among said plurality of computed vigilance parameters.

4. A neural network coupled to perform adaptive pattern recognition, said neural network comprising:

vigilance parameter adjustment means for selectively adjusting a predetermined reference vigilance parameter in response to comparisons thereof with a plurality of computed vigilance parameters based upon input pattern data representing a pattern, said adjustment resulting in a plurality of adjusted reference vigilance parameters;

memory means for storing said plurality of adjusted reference vigilance parameters; and

pattern classification means for classifying said pattern according to the maximum adjusted reference vigilance parameter among said plurality of adjusted reference vigilance parameters.

5. A neural network coupled to perform adaptive pattern recognition, said neural network comprising:

input means for receiving a plurality of input pattern signals I_i and outputting a plurality of corresponding pattern signals X_i (i=1,2,3, . . . , M), wherein each one of said input pattern signals I_i represents a portion of a pattern;

pattern recognition means for providing a plurality of interim pattern signals V_j (j=1,2,3, . . . , N), wherein each one of said interim pattern signals V_j represents said pattern;

bottom-up coupling means for coupling said plurality of pattern signals X_i to said pattern recognition means, said bottom-up coupling means having coupling coefficients Z_ij ;

computation means coupled to said input means for computing a plurality of vigilance parameters P_cj based substantially upon said plurality of pattern signals X_i ;

top-down coupling means for coupling said pattern signals X_i to said computation means, said top-down coupling means having coupling coefficients Z_ji ;

a plurality of memory circuits coupled to selectively store said plurality of vigilance parameters; and

a pattern classification circuit coupled to generate an output pattern signal representing said pattern based upon said stored plurality of vigilance parameters.

6. A neural network as recited in claim 5 further comprising comparator means for receiving a predetermined reference vigilance parameter P_r and for comparing each one of said plurality of computed vigilance parameters P_cj therewith, and still further comprising adjustment means for selectively adjusting said reference vigilance parameter P_r in response to each one of said comparisons thereof with said plurality of computed vigilance parameters P_cj.

7. A neural network as recited in claim 5, wherein said input means comprises a plurality of M nodes and said input pattern signals I_i comprise analog signals.

8. A neural network as recited in claim 5, wherein said input means comprises a plurality of M digital registers and said input pattern signals I_i comprise digital signals.

9. A neural network as recited in claim 5, wherein said pattern recognition means comprises a plurality of N nodes and said interim pattern signals V_j comprise analog signals.

10. A neural network as recited in claim 5, wherein said pattern recognition means comprises a plurality of N digital registers and said interim pattern signals V_j comprise digital signals.

11. A neural network as recited in claim 5, wherein said bottom-up coupling means comprises a bottom-up adaptive weight matrix having a plurality of transistors coupled to selectively weight said plurality of pattern signals X_i, wherein the respective ratios of said weighted pattern signals to corresponding ones of said plurality of pattern signals X_i represent said coupling coefficients Z_ij.

12. A neural network as recited in claim 5, wherein said bottom-up coupling means comprises a bottom-up adaptive weight matrix having a plurality of resistors coupled to selectively weight said plurality of pattern signals X_i, wherein the respective ratios of said weighted pattern signals to corresponding ones of said plurality of pattern signals X_i represent said coupling coefficients Z_ij.

13. A neural network as recited in claim 5, wherein said bottom-up coupling means comprises a bottom-up adaptive weight matrix having a plurality of digital circuits coupled to selectively weight said plurality of pattern signals X_i, wherein the respective ratios of said weighted pattern signals to corresponding ones of said plurality of pattern signals X_i represent said coupling coefficients Z_ij.

14. A neural network as recited in claim 5, wherein said top-down coupling means comprises a top-down adaptive weight matrix having a plurality of transistors coupled to selectively weight said plurality of pattern signals X_i, wherein the respective ratios of said weighted pattern signals to corresponding ones of said plurality of pattern signals X_i represent said coupling coefficients Z_ji.

15. A neural network as recited in claim 5, wherein said top-down coupling means comprises a top-down adaptive weight matrix having a plurality of resistors coupled to selectively weight said plurality of pattern signals X_i, wherein the respective ratios of said weighted pattern signals to corresponding ones of said plurality of pattern signals X_i represent said coupling coefficients Z_ji.

16. A neural network as recited in claim 5, wherein said top-down coupling means comprises a top-down adaptive weight matrix having a plurality of digital circuits coupled to selectively weight said plurality of pattern signals X_i, wherein the respective ratios of said weighted pattern signals to corresponding ones of said plurality of pattern signals X_i represent said coupling coefficients Z_ji.

17. A neural network as recited in claim 5, wherein said computation means comprises an analog signal summing circuit and said pattern signals X_i comprise analog signals, said analog signal summing circuit operating according to the formula:

##EQU10##

18. A neural network as recited in claim 5, wherein said computation means comprises a digital adder circuit and said pattern signals X_i comprise digital signals, said digital adder circuit operating according to the formula:

##EQU11##

19. A neural network as recited in claim 6, wherein said comparator means comprises an analog comparator circuit, and said computed vigilance parameter P_cj and said predetermined reference vigilance parameter P_r comprise voltages, said analog comparator circuit being coupled to determine whether said computed vigilance parameter P_cj equals or exceeds said predetermined reference vigilance parameter P_r and operating according to the formula:

##EQU12##

20. A neural network as recited in claim 6, wherein said comparator means comprises a digital comparator circuit, and said computed vigilance parameter P_cj and said predetermined reference vigilance parameter P_r comprise digital signals, said digital comparator circuit being coupled to determine whether said computed vigilance parameter P_cj equals or exceeds said predetermined reference vigilance parameter P_r and operating according to the formula:

##EQU13##

21. A neural network as recited in claim 6, wherein said adjustment means comprises an analog signal summing circuit, and said reference vigilance parameter P_r and said computed vigilance parameter P_cj comprise analog signals.

22. A neural network as recited in claim 6, wherein said adjustment means comprises a digital adder circuit, and said reference vigilance parameter P_r and said computed vigilance parameter P_cj comprise digital signals.

23. A method of performing adaptive pattern recognition using a self-organizing neural network, said neural network comprising a plurality of M input neurons coupled to accept input pattern data signals I_i (i=1,2,3, . . . , M) representing a pattern and a plurality of N output neurons coupled to provide pattern output signals V_j (j=1,2,3, . . . , N) corresponding to said pattern, said neural network further comprising a bottom-up adaptive weight matrix having coefficients Z_ij and coupling said input neurons to said output neurons, and a top-down adaptive weight matrix having coefficients Z_ji and coupling said output neurons to said input neurons, said neural network still further comprising vigilance parameter computing means for computing a plurality of vigilance parameters P_cj based substantially upon said input pattern data signals I_i and said bottom-up adaptive weight matrix coefficients Z_ij, and vigilance parameter comparator means for individually comparing said plurality of computed vigilance parameters P_cj with a predetermined reference vigilance parameter P_r, said method comprising the steps of:

training said neural network by inputting a plurality of reference pattern data signals I_ir representing a plurality of reference patterns into said plurality of M input neurons;

analyzing a subject pattern by inputting pattern data signals I_i corresponding thereto into said plurality of M input neurons; and

classifying said subject pattern according to a pattern output signal V_jm from the one of said plurality of N output neurons which corresponds to an associated vigilance parameter P_ajm having the maximum value among a plurality of associated vigilance parameters P_aj, said step of classifying said subject pattern comprising outputting an output pattern signal encoded to represent said subject pattern.

24. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 23, wherein said step of training said neural network comprises the steps of:

initializing said coefficients Z_ij of said bottom-up adaptive weight matrix, said coefficients Z_ji of said top-down adaptive weight matrix, and said value P_r of said reference vigilance parameter;

applying said reference input pattern data signals I_ir representing a reference pattern to said plurality of M input neurons, selectively scaling said reference input pattern data signals I_ir by a predetermined scaling factor, and generating corresponding reference pattern signals X_ir ;

generating within each one of said output neurons a plurality of reference pattern output signals V_jr (j=1,2,3, . . . , N) corresponding to said reference pattern;

selecting a reference pattern output signal V_jrm, said selected reference pattern output signal V_jrm corresponding to the one of said output neurons which best represents said reference pattern;

computing a vigilance parameter P_cjm, said computed vigilance parameter P_cjm being associated with said output neuron corresponding to said selected reference pattern output signal V_jrm ;

comparing said computed vigilance parameter P_cjm to said reference vigilance parameter P_r ;

if P_cjm <

P_r, disabling temporarily said output neuron corresponding to said selected reference pattern output signal V_jrm, selecting a new reference pattern output signal V_jrm, and repeating the foregoing steps, beginning with said step of computing a vigilance parameter P_cjm ;

if P_cjm ≧

P_r, adapting the corresponding ones of said coefficients Z_ij, Z_ji of said bottom-up and top-down adaptive weight matrices; and

enabling any output neurons previously disabled in the foregoing disabling step and repeating the foregoing steps, beginning with said step of applying reference input pattern data signals I_ir.

25. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 24, wherein said step of initializing comprises the steps of:

selectively initializing said coefficients Z_ij of said bottom-up adaptive weight matrix according to the formula;

##EQU14## initializing said coefficients Z_ji of said top-down adaptive weight matrix according to the formula;

##EQU15## and selectively initializing said value P_r of said reference vigilance parameter according to the formula;

space="preserve" listing-type="equation"><

P.sub.r <

1.

26. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 24, wherein said step of generating a plurality of reference pattern output signals V_jr is performed according to the formula:

##EQU16##

27. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 24, wherein said step of selecting a reference pattern output signal V_jrm comprises selecting the reference pattern output signal V_jrm having the maximum value among all of said reference pattern output signals V_jr according to the formula:

##EQU17## by inhibiting all other output neurons not corresponding thereto.

28. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 24, wherein said step of computing said computed vigilance parameter P_cjm is performed according to the formula:

##EQU18##

29. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 24, wherein said step of adapting said coefficients Z_ij, Z_ji of said bottom-up and top-down adaptive weight matrices is performed according to the formulas:

##EQU19##

30. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 23, wherein said step of analyzing a subject pattern comprises the steps of:

applying said input pattern data signals I_i representing a subject pattern to said plurality of M input neurons, selectively scaling said input pattern data signals I_i by a predetermined scaling factor, and generating corresponding pattern signals X_i ;

computing said plurality of computed vigilance parameters P_cj, each one of said plurality of computed vigilance parameters P_cj being respectively associated with one of said plurality of N output neurons; and

selectively storing said plurality of computed vigilance parameters P_cj in a random access memory as said plurality of associated vigilance parameters P_aj.

31. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 30, wherein said step of computing said plurality of computed vigilance parameters P_cj is performed according to the formula:

##EQU20##

32. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 30 further comprising the steps of:

re-initializing said value P_r of said reference vigilance parameter according to the formula;

space="preserve" listing-type="equation">P.sub.r =1;

comparing said plurality of computed vigilance parameters P_cj to said reference vigilance parameter P_r ;

if P_cj <

P_r, reducing said value P_r of said reference vigilance parameter by a selectable amount Δ

, re-comparing said computed vigilance parameter P_cj to said reduced reference vigilance parameter P_rrj (where P_rrj =P_r -Δ

), and continuing to reduce said reduced reference vigilance parameter P_rrj until P_cj ≧

P_rrj ;

if P_cj ≧

P_r, storing said reference vigilance parameter P_r in a random access memory as said associated vigilance parameter P_aj ;

if P_cj ≧

P_rrj, storing said reduced reference vigilance parameter P_rrj as said associated vigilance parameter P_aj in a random access memory; and

repeating the foregoing steps until each one of said plurality of N output neurons has associated respectively therewith an associated vigilance parameter P_aj.

33. A method of performing adaptive pattern recognition using a self-organizing neural network as recited in claim 32, wherein said step of computing said plurality of computed vigilance parameters P_cj is performed according to the formula:

##EQU21##