System and method for cognitive memory and auto-associative neural network based pattern recognition

• US 7,702,599 B2
• Filed: 10/07/2005
• Issued: 04/20/2010
• Est. Priority Date: 10/07/2004
• Status: Expired due to Fees

• Current Assignee: Bernard Widrow
• Original Assignee: Bernard Widrow
• Inventors: Widrow, Bernard
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): BUSS, BENJAMIN J
• Application Number:
  US11/245,861
• Publication Number:
  US 20060173560A1
• Time in Patent Office: 1,656 Days
• Field of Search: 706/18, 706/20, 706/934, 706/15, 706/16, 706/25, 706/47, 706/52, 706/924, 382/31, 382/32, 382/156, 382/159, 382/190, 382/224, 359/107, 359/108, 359/559, 359/561
• US Class Current: 706/26
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06V 10/70 using pattern recognition or machine learning optical pattern recognition or electronic computations therefor G06V10/88
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: Y10S 706/934 Information retrieval or Information management

Abstract

Designs for cognitive memory systems storing input data, images, or patterns, and retrieving it without knowledge of where stored when cognitive memory is prompted by query pattern that is related to sought stored pattern. Retrieval system of cognitive memory uses autoassociative neural networks and techniques for pre-processing query pattern to establish relationship between query pattern and sought stored pattern, to locate sought pattern, and to retrieve it and ancillary data. Cognitive memory, when connected to computer or information appliance introduces computational architecture that applies to systems and methods for navigation, location and recognition of objects in images, character recognition, facial recognition, medical analysis and diagnosis, video image analysis, and to photographic search engines that when prompted with a query photograph containing faces and objects will retrieve related photographs stored in computer or other information appliance, and will identify URL'"'"'s of related photographs and documents stored on the World Wide Web.

Claims

1. A cognitive memory system configured for receiving input data or input patterns from one or more sensors, storing said input data or input patterns wherever storage space is available, and retrieving said input data or input patterns upon receipt of an input prompt or query pattern, wherein said cognitive memory system comprises:

(a) a memory input line that carries said input data and input patterns throughout said cognitive memory system for recording wherever storage space is available;

(b) a prompt line that carries said input prompt or query pattern throughout said cognitive memory system;

(c) one or more memory segments, each memory segment having;

(i) one or more memory folders, each memory folder configured for storing said input data or input patterns from said one or more sensors streaming in a sequence over time, and a scanner for continuously scanning the stored contents of said one or more folders and generating patterns;

(ii) a trainable auto-associative neural network whose training input patterns are obtained from said scanner generated patterns and whose sensing input patterns are said input query patterns obtained from said prompt line, said trainable auto-associative neural network further including;

a training algorithm for training said auto-associative neural network to reproduce said training input patterns at its output;

(iii) a first comparator configured for pixel-by-pixel subtracting said sensing input patterns from the auto-associative neural network output to form first error patterns,

(iv) first means for computing the magnitude or mean square of the first error patterns,

(v) a first threshold device that closes a first switch when a sensing input pattern is identified as a hit query pattern when the magnitude or mean square of its first error pattern is below the first threshold level of said first threshold device, said first switch configured to connect a prompt memory buffer to said prompt line to store said hit query pattern in said prompt memory buffer;

(vi) a second comparator configured for pixel-by-pixel subtraction of said scanner generated patterns received from said scanner from said hit query pattern stored in said prompt memory buffer and generating differences, the generated differences being second error patterns;

(vii) second means for computing the magnitude or mean square of the second error patterns,

(viii) a second threshold device that closes a second switch when there is a second hit when the magnitude or mean square of a second error pattern is below the second threshold level of said second threshold device, said second switch configured to connect the memory output line of said memory segment to deliver as output the contents of the memory folder containing the hit pattern associated with said second hit.

2. The cognitive memory system of claim 1, wherein said autoassociative neural network is a multilayer perceptron trained by the back-propagation algorithm and implemented by means of parallel digital hardware.

3. The cognitive memory system of claim 1, wherein input prompt or input query patterns are obtained from said contents of the memory folder containing said hit pattern associated with said second hit.

4. The cognitive memory system of claim 1, wherein said input data comprises visual or photographic images, said input data being first pre-processed by any one of or any combination of rotation, translation, scaling, varying brightness and contrast before carried throughout said cognitive memory system by said memory input line.

5. The cognitive memory system of claim 1, wherein visual or photographic input prompt or query patterns are first pre-processed by a combination of rotation, translation, scaling, varying brightness and contrast before being carried throughout said cognitive memory system by said prompt line.

6. The cognitive memory system of claim 1, used for face detection wherein:

(a) said input data or patterns are facial images of one or more individual persons, each facial image containing 2,000 or less pixels;

(b) said input query patterns are obtained by scanning with a variable window images or photographs containing faces of one or more individuals, said variable window containing 2,000 or less pixels, said variable window capable of variation by any combination of rotation, translation, scaling, brightness, and contrast; and

(c) said variable window outlines a detected face when there is a said second hit, when a pattern or patterns appear on the memory output line.

7. The cognitive memory system of claim 1, used for face recognition wherein:

(a) said input data or patterns are one or more facial images of known individual persons and their respective identities;

(b) said input query patterns are obtained by scanning images or photographs with a variable window, said variable window capable of variation by any combination of rotation, translation, scaling, brightness, and contrast; and

(c) said variable window outlines an identifiable face when there is a said second hit, when a pattern or patterns appear on the memory output line with their identification.

8. The cognitive memory system of claim 1, used for processing auditory signals wherein:

(a) said input data or patterns are obtained from auditory inputs subject to any one of or any combination of time dilation, time contraction, time warping, amplitude scaling, filtering, spectral analysis, and translation in time; and

(b) said input query patterns are obtained from auditory inputs subject to any combination of time dilation, time contraction, time warping, amplitude scaling, filtering, spectral analysis, and translation in time.

9. The cognitive memory system of claim 1, wherein:

said autoassociative neural network is a multilayer perceptron trained by the back-propagation algorithm and implemented by means of parallel digital hardware;

said input prompt or input query patterns are obtained from said contents of the memory folder containing said hit pattern associated with said second hit;

said input data comprises visual or photographic images, said input data being first pre-processed by any one of or any combination of rotation, translation, scaling, varying brightness and contrast before carried throughout said cognitive memory system by said memory input line;

visual or photographic input prompt or query patterns are first pre-processed by a combination of rotation, translation, scaling, varying brightness and contrast before being carried throughout said cognitive memory system by said prompt line; and

said cognitive memory system is used for face recognition wherein;

said input data or patterns are one or more facial images of known individual persons and their respective identities;

said input query patterns are obtained by scanning images or photographs with a variable window, said variable window capable of variation by any combination of rotation, translation, scaling, brightness, and contrast; and

said variable window outlines an identifiable face when there is said second hit, when a pattern or patterns appear on the memory output line with their identification.

10. A method for storing and retrieving input data or input patterns with a cognitive memory system having computer readable memory folders on a computer or other information appliance, said input data or input patterns received from one or more sensors, said data or input patterns stored wherever space is available in said memory folders, each memory folder configured for storing a plurality of patterns from said one or more sensors and configured for storing other identifying or related data, said method adapted for retrieving the contents of one or more of said memory folders containing one or more stored patterns related to an input prompt or query pattern, said method comprising the steps of:

(a) carrying said input data or input patterns throughout said cognitive memory system using a memory input line, and storing said input data or input patterns wherever space is available in said memory folders coupled to said memory input line;

(b) carrying said input prompt or query pattern throughout said cognitive memory system using a prompt line;

(c) organizing said cognitive memory system into one or more segments, each of the one or more segments comprised of one or more of said memory folders, a scanner, a prompt memory buffer, an autoassociative neural network, a first comparator, a first threshold, a first switch, a second comparator, a second threshold, a second switch, and a memory output line;

(d) generating training patterns in each segment by scanning its memory folders with its scanner, and training its autoassociative neural network with said training patterns;

(e) sensing the cognitive memory system by applying said input prompt or query pattern to said prompt line;

(f) deriving a sensing input pattern for each autoassociative neural network in each segment from said prompt line;

(g) sensing each autoassociative neural network in each segment by comparing with said first comparator the input patterns and output patterns of said autoassociative neural network when the input pattern is said input prompt pattern or query pattern;

(h) closing a first switch in each segment where there is a first hit, when the first comparator error is below the level of said first threshold, said first switch connecting said prompt memory buffer to said prompt line;

(i) generating patterns by scanning the memory folders of each segment and comparing using said second comparator these scanned patterns with the pattern stored in said prompt memory buffer of said each segment; and

(j) closing a second switch in each hit segment where there is a second hit, where the second comparator error is below the threshold level of said second threshold, said second switch configured to connect the memory output line of said hit segment to the hit memory folder containing the pattern matching the pattern stored in the prompt memory buffer, delivering the pattern contents of said hit folder as the output of said hit segment.

11. The method of claim 10, further comprising:

using autoassociative neural networks including multilayer perceptrons; and

training said autoassociative neural networks using a back-propagation algorithm.

12. The method of claim 10, further comprising:

using said contents retrieved from a said hit memory folder to serve as self-prompt input query patterns, and initiating further retrieval of contents of additional memory folders.

13. The method of claim 10, further comprising:

preprocessing said input data or input patterns by any one of or any combination of rotation, translation, scaling, varying brightness, and contrast before carrying throughout said cognitive memory system by said memory input line.

14. The method of claim 10, further comprising:

preprocessing said input prompt or query pattern by any one of or combination of rotation, translation, scaling, varying brightness and contrast before carrying throughout said cognitive memory system via said prompt line.

15. The method of claim 10, wherein:

said method used for face detection and said input data or input patterns include faces, and said method further comprising;

(a) obtaining said input data or input patterns as individual facial images of one or more persons, each individual facial image containing 2,000 or less pixels;

(b) obtaining said input prompt or query patterns by scanning with a variable window images or photographs containing faces of one or more persons, said variable window containing 2,000 or less pixels, said variable window capable of variation by any combination of rotation, translation, scaling, brightness and contrast; and

(c) outlining a detected face by said variable window when there is a said second hit, when a pattern or patterns appear on said memory output line of one or more of said hit segments.

16. The method of claim 10, said wherein:

said method used for face recognition and said input data or input patterns include faces, and said method further comprising;

(a) obtaining said input data or input patterns as one or more individual facial images of one or more known persons and their respective identities;

(b) obtaining said input prompt or query patterns by scanning with a variable window images or photographs containing faces of one or more people, said variable window capable of variation by any combination of rotation, translation, scaling, brightness and contrast; and

(c) identifying and outlining a known face when there is a said second hit, when a pattern or patterns appear with identification on said memory output line of one or more of said hit segments.

17. The method of claim 10, wherein said method is used for processing auditory signals and said input data or input patterns include auditory signals, and said method further comprising:

(a) obtaining said input data or input patterns from auditory inputs subject to any one or any combination of time dilation, time contraction, time warping, amplitude scaling, filtering, spectral analysis, and translation in time; and

(b) obtaining said input query patterns from auditory inputs subject to any combination of time dilation, time contraction, time warping, amplitude scaling, filtering, spectral analysis, and translation in time.

18. The method of claim 10, further comprising:

using autoassociative neural networks including multilayer perceptrons, and training said autoassociative neural networks using a back-propagation algorithm;

using said contents retrieved from a said hit memory folder to serve as self-prompt input query patterns, and initiating further retrieval of contents of additional memory folders;

preprocessing said input data or input patterns by any one of or any combination of rotation, translation, scaling, varying brightness, and contrast before carrying throughout said cognitive memory system by said memory input line;

preprocessing said input prompt or query pattern by any one of or combination of rotation, translation, scaling, varying brightness and contrast before carrying throughout said cognitive memory system via said prompt line; and

wherein;

said method is used for face detection and said input data or input patterns include faces, and said method further comprising;

obtaining said input data or input patterns as one or more individual facial images of one or more known persons and their respective identities;

obtaining said input prompt or query patterns by scanning with a variable window images or photographs containing faces of one or more people, said variable window capable of variation by any combination of rotation, translation, scaling, brightness and contrast; and

identifying and outlining a known face when there is a said second hit, when a pattern or patterns appear with identification on said memory output line of one or more of said hit segments.

19. The method of claim 10, wherein:

said method used for face recognition and for processing auditory signals and said input data or input patterns include faces and auditory signals, and said method further comprising;

(a) obtaining said input data or input patterns as;

(i) one or more individual facial images of one or more known persons and their respective identities, or (ii) auditory inputs subject to any one or any combination of time dilation, time contraction, time warping, amplitude scaling, filtering, spectral analysis, and translation in time;

(b) obtaining said input prompt or query patterns (iii) for said facial images by scanning with a variable window images or photographs containing faces of one or more people, said variable window capable of variation by any combination of rotation, translation, scaling, brightness and contrast, and (iv) for said auditory inputs from auditory inputs subject to any combination of time dilation, time contraction, time warping, amplitude scaling, filtering, spectral analysis, and translation in time; and

(c) identifying and outlining a known face when there is a said second hit, when a pattern or patterns appear with identification on said memory output line of one or more of said hit segments; and

identifying a known auditory signal when there is a second auditory hit.