System for self-organization of stable category recognition codes for analog input patterns

• US 4,914,708 A
• Filed: 06/19/1987
• Issued: 04/03/1990
• Est. Priority Date: 06/19/1987
• Status: Expired due to Term

• Current Assignee: Boston University
• Original Assignee: Boston University
• Inventors: Grossberg, Stephen, Carpenter, Gail A.
• Primary Examiner(s): Blum, Theodore M.
• Application Number:
  US07/064,764
• Time in Patent Office: 1,019 Days
• Field of Search: 382/14, 382/15
• US Class Current: 382/156
• CPC Class Codes: G06N 3/0409 Adaptive resonance theory [ART] networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

A neural network includes a feature representation field which receives input patterns. Signals from the feature representation field select a category from a category representation field through a first adaptive filter. Based on the selected category, a template pattern is applied to the feature representation field, and a match between the template and the input is determined. If the angle between the template vector and a vector within the representation field is too great, the selected category is reset. Otherwise the category selection and template pattern are adapted to the input pattern as well as the previously stored template. A complex representation field includes signals normalized relative to signals across the field and feedback for pattern contrast enhancement.

Claims

1. In a pattern recognition device comprising a short term feature representation field for receiving input signals defining an input pattern and template signals defining a long term memory template, means for selecting at least one category node based on a pattern from the short term field, means for generating the long term template based on the selected category and means for adapting category selection and the long term memory template to the input signals, a category selection being reset with an insufficient match between the input pattern and template, the improvement wherein the match is based on the closeness between a vector of the template and a vector of the short term field.

2. A device as claimed in claim 1 wherein the closeness of the two vectors is disregarded where the template is not based on previous input patterns such that the category selection is not reset where a category has not been previously selected.

3. A device as claimed in claim 2 wherein the match is based on a norm of weighted sums, each sum being of a short term signal of the short term field and a template signal.

4. A device as claimed in claim 3 wherein the summed short term signal is normalized relative to other like signals of the short term field.

5. A device as claimed in claim 4 further comprising means for enhancing higher level signals of the input relative to lower level signals.

6. A device as claimed in claim 5 further comprising feedback of enhanced signals within the short term field.

7. A device as claimed in claim 6 further comprising feedback of the weighted sum of the short term signal and template signal within the short term field.

8. A device as claimed in claim 7 wherein the short term field comprises matched signals of a matched pattern, each matched signal being the weighted sum of a short term signal in the field and a template signal, the category selection and long term memory template being adapted as dual functions of the difference between the template signal and the matched signal.

9. A device as claimed in claim 2 wherein the short term field comprises matched signals of a matched pattern, each matched signal being the weighted sum of a short term signal in the field and a template signal, the category selection and the long term memory template being adapted as dual functions of the difference between the template signal and the matched signal.

10. A device as claimed in claim 9 wherein the short term signal summed with the template signal is normalized relative to other like signals of the short term field.

11. A device as claimed in claim 2 further comprising means for enhancing higher level signals of the input relative to lower level signals.

12. A device as claimed in claim 11 further comprising feedback of enhanced signals within the short term field.

13. A device as claimed in claim 12 further comprising feedback of template signals within the short term field.

14. A device as claimed in claim 2 wherein the input pattern to the system is an analog pattern.

15. A device as claimed in claim 2 wherein closeness between the vectors is determined from the angle between the vectors.

16. In a pattern recognition device comprising a short term feature representation field for receiving input signals defining an input pattern and template signals defining a long term memory template, means for selecting at least one category node based on a pattern from the short term field, means for generating the long term template based on the selected category and means for adapting category selection and the long term memory template to the input signals, the improvement wherein the short term field comprises a plurality of subfields of short term signals and feedback of short term signals within the short term field.

17. A device as claimed in claim 16 further comprising means for enhancing higher level signals of the input relative to lower level signals.

18. A device as claimed in claim 17 further comprising feedback of enhanced signals within the short term field.

19. A device as claimed in claim 18 further comprising feedback of template signals within the short term field.

20. A device as claimed in claim 16 wherein a feedback signal is normalized relative to other like signals of the short term field.

21. A pattern recognition device comprising a short term feature representation field for receiving input signals defining an input pattern and template signals defining a long term memory template, means for selecting at least one category node based on a pattern from the short term field, means for generating the long term memory template based on the selected category and means for adapting category selection and a long term memory template to the input signals, the improvement wherein the short term field comprises matched signals of a matched pattern, each matched signal being the weighted sum of a short term signal in the field and a template signal, the category selection and the long term memory template being adapted as dual functions of the difference between a template signal and the matched signal.

22. A device as claimed in claim 21 wherein the summed short term signal is normalized relative to other like signals of the short term field.

23. A device as claimed in claim 21 wherein a category selection is reset with an insufficient match between the input pattern and template, the match being based on the closeness between the vector of the template and a vector of the short term field as indicated by an angle between the two vectors.

24. A device as claimed in claim 23 wherein the summed short term signal is normalized relative to other like signals of the short term field.

25. A device as claimed in claim 24 further comprising means for enhancing higher level signals of the input relative to lower level signals.

26. A device as claimed in claim 23 wherein closeness between the vectors is determined from the angle between the vectors.

27. In a pattern recognition device comprising a short term feature representation field for receiving input signals defining an input pattern and template signals defining a long term memory template, means for selecting at least one category based on a pattern from the short term field, means for generating the long term memory template based on the selected category, and means for adapting category selection and the long term memory template to the input signals, the improvement wherein the short term field comprises a plurality of subfields of short term signals, each of the short term signals of at least one subfield being based on an input signal specific to the short term signal and also on non-specific signals generated from across the short term field.

28. A device as claimed in claim 27 wherein each short term signal of a subfield is normalized relative to other signals of the subfield.

29. A device as claimed in claim 28 wherein higher level signals of the normalized subfield are enhanced relative to lower level signals of the subfield.

30. A device as claimed in claim 29 further comprising feedback of an enhanced normalized input pattern to modify the input pattern to the system.

31. A device as claimed in claim 28 wherein a category selection is reset with an insufficient match between the input pattern and template, the match being based on a norm of weighted sums, each sum being of a normalized short term signal and a template signal.

32. A device as claimed in claim 31 further comprising feedback of the weighted sum to modify the normalized short term signal.

33. A device as claimed in claim 27 wherein a category selection is reset with an insufficient match between the input pattern and template, the match being based on a norm of weighted sums, each sum being of a normalized short term signal and a template signal.

34. A device as claimed in claim 33 wherein each short term signal of a subfield is normalized relative to other signals of the subfield.

35. A device as claimed in claim 27 having the ability to self-organize stable recognition categories at variable levels of vigilance in response to arbitrary temporal lists of analog or digital image patterns to learn invariant pattern recognition codes in response to noisy image data, the device having a preprocessor for generating the input pattern, the preprocessor comprising means for automatic separation of image figure of the image pattern from ground, means for sharpening and completing the resultant image boundaries, and means for processing the resultant image using an invariant filter.

36. A device as claimed in claim 35 wherein the means for sharpening and completing performs a boundary contour technique.

37. A device as claimed in claim 35 wherein the invariant filter is a Fourier-Mellon filter.

38. A pattern recognition method comprising:

receiving in a short term feature representation field input signals defining an input pattern and template signals defining a long term memory template;

selecting at least one category node based on a pattern from the short term field, category selection being reset with an insufficient match between the input pattern and template based on the closeness between a vector of the template and a vector of the short term field;

generating the long term template based on the selected category which is not reset; and

adapting category selection and the long term memory template to the input signals.

39. A method as claimed in claim 38 wherein the closeness of the two vectors is disregarded where the template is not based on previous input patterns such that the category selection is not reset where a category has not been previously selected.

40. A method as claimed in claim 39 wherein the match is based on a norm of weighted sums, each sum being of a short term signal of the short term field and a template signal.

41. A method as claimed in claim 40 wherein the summed short term signal is normalized relative to other like signals of the short term field.

42. A method as claimed in claim 41 further comprising enhancing higher level signals of the input relative to lower level signals.

43. A method as claimed in claim 42 further comprising feeding enhanced signals back within the short term field.

44. A method as claimed in claim 43 further comprising feeding the weighted sum of the short term signal and template signal back within the short term field.

45. A method as claimed in claim 44 wherein the short term field comprises matched signals of a matched pattern, each matched signal being the weighted sum of a short term signal in the field and a template signal, the category selection and long term memory template being adapted as dual functions of the difference between the template signal and the matched signal.

46. A method as claimed in claim 39 wherein the short term field comprises matched signals of a matched pattern, each matched signal being the weighted sum of a short term signal in the field and a template signal, the category selection and the long term memory template being adapted as dual functions of the difference between the template signal and the matched signal.

47. A method as claimed in claim 46 wherein the short term signal summed with the template signal is normalized relative to other like signals of the short term field.

48. A method as claimed in claim 39 further comprising enhancing higher level signals of the input relative to lower level signals.

49. A method as claimed in claim 48 further comprising feeding enhanced signals back within the short term field.

50. A method as claimed in claim 49 further comprising feeding template signals back within the short term field.

51. A method as claimed in claim 39 wherein the input pattern to the system is an analog pattern.

52. A method as claimed in claim 39 wherein closeness between the vectors is determined from the angle between the vectors.

53. A pattern recognition method comprising:

receiving in a short term feature representation field comprising a plurality of subfields of short term signals input signals defining an input pattern and template signals defining a long term memory template and feeding short term signals back within the short term fields;

selecting at least one category node based on a pattern from the short term field;

generating the long term template based on the selected category; and

adapting category selection and the long term memory template to the input signals and feedback of short term signals within the short term field.

54. A method as claimed in claim 53 further comprising enhancing higher level signals of the input relative to lower level signals.

55. A method as claimed in claim 54 further comprising feeding enhanced signals back within the short term field.

56. A method as claimed in claim 55 further comprising feeding template signals back within the short term field.

57. A method as claimed in claim 56 wherein a feedback signal is normalized relative to other like signals of the short term field.

58. A pattern recognition method comprising:

receiving in a short term feature representation field input signals defining a long term memory template, the short term field comprising matched signals of a matched pattern, each matched signal being the weighted sum of a short term signal in the filed and a template signal;

selecting at least one category node based on a pattern from the short term field;

generating the long term memory template based on the selected category; and

adapting category selection and a long term memory template to the input signals as dual functions of the difference between a template signal and the matched signal.

59. A method as claimed in claim 58 wherein the summed short term signal is normalized relative to other like signals of the short term field.

60. A method as claimed in claim 58 wherein a category selection is reset with an insufficient match between the input pattern and template, the match being based on the closeness between the vector of the template and a vector of the short term field as indicated by an angle between the two vectors.

61. A method as claimed in claim 60 wherein the summed short term signal is normalized relative to other like signals of the short term field.

62. A method as claimed in claim 61 further comprising means for enhancing higher level signals of the input relative to lower level signals.

63. A method as claimed in claim 60 wherein closeness between the vectors is determined from the angle between the vectors.

64. A pattern recognition method comprising:

receiving in a short term feature representation field input signals defining an input pattern and template signals defining a long term memory template, the short term field comprising a plurality of subfields of short term signals, each of the short term signals of at least one subfield being based on an input signal specific to the short term signal and also on non-specific signals generated from across the short term field;

selecting at least one category based on a pattern from the short term field;

generating the long term memory template based on the selected category; and

adapting category selection and the long term memory template to the input signals.

65. A method as claimed in claim 64 wherein each short term signal of a subfield is normalized relative to other signals of the subfield.

66. A method as claimed in claim 65 wherein higher level signals of the normalized subfield are enhanced relative to lower level signals of the subfield.

67. A method as claimed in claim 66 further comprising feeding an enhanced normalized input pattern back to modify the input pattern to the system.

68. A method as claimed in claim 65 wherein a category selection is reset with an insufficient match between the input pattern and template, the match being based on a norm of weighted sums, each sum being of a normalized short term signal and a template signal.

69. A method as claimed in claim 68 further comprising feeding the weighted sum back to modify the normalized short term signal.

70. A method as claimed in claim 64 wherein a category selection is reset with an insufficient match between the input pattern and template, the match being based on a norm of weighted sums, each sum being of a normalized short term signal and a template signal.

71. A method as claimed in claim 70 wherein each short term signal of a subfield is normalized relative to other signals of the subfield.

72. A method as claimed in claim 64 comprising preprocessing a pattern to generate the input pattern, the preprocessing comprising separating image figure of the image pattern from ground, sharpening and completing the resultant image boundaries, and processing the resultant image using an invariant filter.

73. A method as claimed in claim 72 wherein the sharpening and completing is by a boundary contour technique.

74. A method as claimed in claim 72 wherein the invariant filter is a Fourier-Mellon filter.