SYSTEM AND METHOD FOR COGNITIVE MEMORY AND AUTO-ASSOCIATIVE NEURAL NETWORK BASED PATTERN RECOGNITION

• US 20100312734A1
• Filed: 04/16/2010
• Published: 12/09/2010
• Est. Priority Date: 10/07/2005
• Status: Abandoned Application

• Current Assignee: Bernard Widrow
• Original Assignee: Bernard Widrow
• Inventors: Widrow, Bernard
• Application Number:
  US12/762,092
• Publication Number:
  US 20100312734A1
• Time in Patent Office: Days
• US Class Current: 706/25
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/176 Urban or other man-made structures

Abstract

Designs for cognitive memory systems storing input data, images, or patterns, and retrieving it without knowledge of where stored when cognitive memory is prompted by query pattern that is related to sought stored pattern. Retrieval system of cognitive memory uses autoassociative neural networks and techniques for pre-processing query pattern to establish relationship between query pattern and sought stored pattern, to locate sought pattern, and to retrieve it and ancillary data. Cognitive memory, when connected to computer or information appliance introduces computational architecture that applies to systems and methods for navigation, location and recognition of objects in images, character recognition, facial recognition, medical analysis and diagnosis, video image analysis, and to photographic search engines that when prompted with a query photograph containing faces and objects will retrieve related photographs stored in computer or other information appliance, and will identify URL'"'"'s of related photographs and documents stored on the World Wide Web.

Claims

1. A cognitive memory system configured for receiving input data from one or more external sensors, and using said input data upon receipt of an input prompt or query pattern, said cognitive memory system comprising:

a memory segment configured for storing input data received from said external sensor and for generating patterns from said input;

an auto-associative neural network whose training input patterns are obtained from said generated patterns and whose input patterns are input query patterns, said trainable auto-associative neural network further including;

a training algorithm for training said auto-associative neural network to reproduce said training input patterns at its output;

a first comparator configured for an arithmetic operation of said input patterns from the auto-associative neural network output to form first error patterns,

a first threshold device that changes a logic condition when a sensing input pattern is identified as a hit query pattern when a magnitude a first error pattern is below a first threshold level of said first threshold device, said change in logic causes storing of said hit query pattern in a prompt memory element;

a second comparator configured for an arithmetic operation of said generated patterns with said hit query pattern stored in said prompt memory element and generating differences, the generated differences being second error patterns; and

a second threshold device that changes a second logic closes when there is a second hit when a magnitude of a second error pattern is below a second threshold level of said second threshold device, said second change in logic causes delivery as output the contents of the memory element storing the hit pattern associated with said second hit.

2. The cognitive memory system of claim 1, wherein said autoassociative neural network is a multilayer perceptron trained by the back-propagation algorithm and implemented by means of parallel digital hardware.

3. The cognitive memory system of claim 1, wherein input prompt or input query patterns are obtained from contents of the memory element containing hit pattern associated with said second hit.

4. The cognitive memory system of claim 1, wherein said input data comprises photographic or satellite images from any source.

5. The cognitive memory system of claim 1, wherein the system is used for a surveillance activity.

6. The cognitive memory system of claim 5, wherein the surveillance activity comprises operating at least one security checkpoint.

7. The cognitive memory system of claim 6, wherein at least one security checkpoint further comprises:

at least one camera or sensor for obtaining images at a security checkpoint that provide said query image for said system; and

an alarm or notification system generating at least one alarm or notification upon the recognition of a person of interest;

wherein the persons of interest passing through said security checkpoint system are detected and identified by said system.

8. The system as in claim 7, further comprising:

a plurality of said security checkpoint systems;

an intelligence center coupled with the plurality of security checkpoint systems;

a two-way communication system between said plurality of security checkpoint systems and said intelligence center;

at least one database located at or coupled with said intelligence center that contains facial images of persons of interest and identities of said persons of interest; and

a computer or other information appliance located at or coupled with said intelligence center that is capable of training said images of said persons of interest into a central autoassociative neural network, and transmitting the weights, the structure, and the training patterns of said central autoassociative neural network via said means of two-way communication to said security checkpoint systems, whereupon the weights, structure and training patterns of said central autoassociative neural network are copied into the cognitive memories of the detection systems of said security checkpoint systems;

wherein said persons of interest who pass through said security checkpoint systems are detected and identified, whereupon the detection event is transmitted via said means of two-way communication to said intelligence center, and new high-resolution images of said persons of interest that were taken at said security checkpoint are transmitted via said means of two-way communication to said intelligence center to be added to said database for further training.

16. The system in claim 1, wherein the arithmetic operation is a subtraction or difference generating operation.

19. The surveillance system of claim 16, further comprising:

a human facial recognition system for recognizing persons'"'"' faces depicted in a query image comprising;

memory elements of a computer or other information appliance, said memory elements storing images of faces of interest, and further storing the respective identities of said faces;

an autoassociative neural network trained on said images of faces of interest;

a window that scans over said query image, performing any combination of the following transformations;

rotation, translation, changes in scale, changes in brightness, changes in contrast, spatial filtering, frequency filtering, spatial frequency filtering, edge detection, perspective transformation, warping, distorting, distortion correction, image-to-image registration, gray-level histogram modification or equalization, adjusting color characteristics, varying or adjusting color saturation, removing color, distending, compressing, squeezing, shearing, and changes in intensity;

wherein said window provides prompt input patterns to said autoassociative neural network;

an error measurement unit for measuring the error between said prompt input patterns and resulting output patterns when said prompt input patterns are presented as inputs to said autoassociative neural network;

a first comparator for comparing said error to a pre-set threshold;

a pattern selector for selecting successful prompt patterns, wherein a successful prompt pattern is a prompt input pattern whose error is less than said pre-set threshold;

a second comparator for comparing on a pixel-by-pixel basis said successful prompt patterns with the images stored in said memory elements;

a memory element selector for selecting a hit memory elements, wherein a hit memory element is a memory element that contains an image that closely matches a successful prompt input pattern; and

an output interface for delivering the contents of said hit element as the output.

9. A computer implemented method for operating a cognitive memory that is configured for receiving input data from one or more external sensors, and using the data upon receipt of an input prompt or query pattern, said method comprising:

defining a memory segment configured for storing input data received from said external sensor and for generating patterns from said input;

training an auto-associative neural network whose training input patterns are obtained from the generated patterns and whose input patterns are input query patterns using a training algorithm to reproduce said training input patterns at its output;

configuring and operating a first comparator to perform an arithmetic operation of said input patterns from the auto-associative neural network output to form first error patterns,

configuring and operating a first threshold logic element that changes a first logic condition when a sensing input pattern is identified as a hit query pattern when a magnitude a first error pattern is below a first threshold level of said first threshold device, said change in logic causes storing of said hit query pattern in a prompt memory element;

configuring and operating a second comparator element to perform for an arithmetic operation of said generated patterns with said hit query pattern stored in said prompt memory element and generating differences, the generated differences being second error patterns; and

configuring and operating a second threshold logic element that changes a second logic condition when there is a second hit when a magnitude of a second error pattern is below a second threshold level of said second threshold device, said second change in logic causes delivery as output the contents of the memory element storing the hit pattern associated with said second hit.

10. The method of claim 9, further comprising:

using autoassociative neural networks including multilayer perceptrons; and

training said autoassociative neural networks using a back-propagation algorithm.

11. The method of claim 9, further comprising:

using said contents retrieved from a said hit memory element to serve as self-prompt input query patterns, and initiating further retrieval of contents of additional memory elements.

12. The method of claim 9, wherein the method is applied to a surveillance activity.

13. The method of claim 10, wherein the surveillance activity comprises operating at least one security checkpoint.

14. The method of claim 9, wherein the operating or the at least one security checkpoint further comprises:

operating at least one camera or sensor for obtaining images at a security checkpoint that provide said query image for said system; and

generating an alarm or notification upon the recognition of a person of interest;

wherein the persons of interest passing through said security checkpoint system are detected and identified by said system.

15. The method as in claim 14, further comprising:

operating a plurality of said security checkpoint systems in a network;

operating an intelligence center coupled with the plurality of security checkpoint systems;

providing a two-way communication capability between said plurality of security checkpoint systems and said intelligence center;

operating at least one database located at or coupled with said intelligence center that contains facial images of persons of interest and identities of said persons of interest; and

operating a computer or other information appliance located at or coupled with said intelligence center that is capable of training said images of said persons of interest into a central autoassociative neural network, and transmitting the weights, the structure, and the training patterns of said central autoassociative neural network via said means of two-way communication to said security checkpoint systems, whereupon the weights, structure and training patterns of said central autoassociative neural network are copied into the cognitive memories of the detection systems of said security checkpoint systems;

wherein said persons of interest who pass through said security checkpoint systems are detected and identified, whereupon the detection event is transmitted via said means of two-way communication to said intelligence center, and new high-resolution images of said persons of interest that were taken at said security checkpoint are transmitted via said means of two-way communication to said intelligence center to be added to said database for further training.

17. The method in claim 9, wherein the arithmetic operation is a subtraction or difference generating operation.

18. A surveillance system comprising:

a plurality of said security checkpoint systems;

an intelligence center coupled with the plurality of security checkpoint systems;

a two-way communication link or system between said plurality of security checkpoint systems and said intelligence center;

at least one database located at or coupled with said intelligence center that contains facial images of persons of interest and identities of said persons of interest; and

a computer or other information appliance located at or coupled with said intelligence center that is capable of training said images of said persons of interest into a central auto-associative neural network, and transmitting the weights, the structure, and the training patterns of said central auto-associative neural network via said means of two-way communication to said security checkpoint systems, whereupon the weights, structure and training patterns of said central auto-associative neural network are copied into the cognitive memories of the detection systems of said security checkpoint systems;

wherein said persons of interest who pass through said security checkpoint systems are detected and identified, whereupon the detection event is transmitted via said means of two-way communication to said intelligence center, and new high-resolution images of said persons of interest that were taken at said security checkpoint are transmitted via said means of two-way communication to said intelligence center to be added to said database for further training.