Neural networks

• US 5,515,477 A
• Filed: 10/20/1993
• Issued: 05/07/1996
• Est. Priority Date: 04/22/1991
• Status: Expired due to Term

• Current Assignee: And Artificial Neural Devices Corporation
• Original Assignee: John Sutherland
• Inventors: Sutherland, John
• Primary Examiner(s): Moore, David K.
• Assistant Examiner(s): HAFIZ, TARIQ R
• Application Number:
  US08/133,196
• Time in Patent Office: 930 Days
• Field of Search: 395/27, 395/21-25, 382/14, 382/15, 382/153-159
• US Class Current: 706/41
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: Y10S 128/925 Neural network

Abstract

The present invention relates to adaptive information processing systems, and in particular to associative memories utilizing confidence-mediated associations, and especially neural network systems comprising an auto-organizational apparatus and processes for dynamically mapping an input onto a semantically congruous and contemporaneously-valid, learned response. In particular the present invention relates to such an associative memory system in which provision is made for improving the congruence between an associative memory, by impressing a desired response on an associative memory mapping based on complex polar values.

Claims

1. An artificial, auto-associative memory storage device for retrievably storing a response together with an associated at least one input analog stimulus, each response and stimulus having respective ones of representative, pre-assigned complex polar values, said device comprising:

a re-writable correlation substrate including a quantum of required physical storage space for storing a addressably mapped, congruous stimulus-response association thereon, wherein said association is thus stored as an initial correlation value which is the outer product of;

the conjugate of the stimulus one of said preassigned values; and

,

the response one of said pre-assigned values, and, whereby said association represents a certain contemporaneous state of association between said input analog stimulus and said response;

a data bus, for conveying data received by or transmitted from said storage device, to and from said substrate, said data bus being connected in data communicating relation with said substrate, whereby said correlation substrate is subsequently addressably re-written responsive to supervisory intervention input delivered through said bus as a congruency-enhancing association between said at least one input analog stimulus and a predetermined desired response, each having respective, representative ones of a pre-assigned subsequent stimuli and desired response complex polar values which are combined as an inner product of said subsequent stimulus value through said initial correlation value to yield a transformed response value having a measurable vector difference value from said desired response value, which vector difference value is in turn used to perform an element by element vector addition to said initial correlation value to yield said enhanced-correlation value;

whereby, said enhanced, mapped association is stored on said substrate by replacing said initial correlation value with said enhanced correlation value, to thereby enhance the addressable information content stored within a quantum of re-written storage space equal in size to said quantum of required space.

2. The memory storage device according to claim 1 wherein said response, and an associated plurality of stimuli arranged as an N×

1 set, are represented by respective ones of said pre-assigned complex polar values, and are addressably mapped as a single pattern of congruous stimulus-response associations on said re-writable correlation substrate, as a correlation set of correlation values of a corresponding N products of the conjugate of a selected one of said response or stimulus values, and the other of said response or stimulus values, wherein said mapped association represents a certain contemporaneous state of association between said input analog stimuli and said response.

3. The memory storage device according to claim 2 wherein an N plurality of temporally sequenced associations, are mapped as a corresponding plurality of mutually temporally unique patterns arranged as a corresponding plurality of respective correlation sets, and are addressable mapped in superposed relation as a set of congruous stimulus-response association on said re-writable substrate, as a correlation matrix containing N elements corresponding to respective ones of summations of correlation values in each of said correlations sets, wherein said plurality of mapped associations stored on said correlation substrate represent a certain most contemporaneous state of association between each of said input analog stimuli and said response.

4. The device according to claim 3, further including a neural processing engine connected in data communicating relation to said memory storage device and operable to receive analog stimulus input data, and assign and manipulate representative complex polar values thereof, and to communicate same to said storage device.

5. The device according to claim 4 wherein said engine comprises a compound neural processing cell architecture.

6. The device according to claim 5 wherein said engine includes selected ones of input buffer cells for storing data received from a processor;

operator cells for receiving input data and performing a predetermined operation on said data, and storing a result generated through said predetermined operation; and

, a neural cell comprising an at least one memory storage device according to claim 1.

7. The device according to claim 5 wherein at least some of said cells are mutually commutatively interconnected.

8. The device according to claim 5 wherein said cells are configured in a feed forward architecture.

9. The device according to claim 5 wherein said cells are configured in a concurrent architecture.

10. The device according to claim 4 wherein said neural engine is a virtual device operable in a general purpose computer.

11. The device according to claim 10 wherein said general purpose computer comprises an at least one hardware processing node, and a data bus, arranged in complex value signal communicating relation, for routing said signals between said processing node and said memory storage device.

12. The device according to claim 10 wherein a plurality of hardware processing nodes are co-operable in conjunction with distributed processing control means.

13. The device according to claim 4 wherein said engine and storage device comprise a pattern classification device.

14. The device according to claim 4 wherein said engine and storage device comprise a spatio-temporal learning system comprising a recurrent network architecture of processing cells.

15. The device according to claim 14 wherein said engine and storage device comprise an analog control signal generation system.

16. The device according to claim 4 wherein said engine and storage device comprise an expert system.

17. The device according to claim 4 wherein said engine and storage device comprise an analog simulation and forecasting system.

18. The device according to claim 4 wherein said engine and storage device comprise a navigational control system.

19. The device according to claim 4 wherein said engine and storage device comprise an automatic target recognition and targeting control system.

20. The device according to claim 4 wherein said engine and storage device comprise a linear associative memory system.

21. The device according to claim 4 wherein said engine and storage device comprise a data compression system.