Concurrent learning and performance information processing system

• US 6,289,330 B1
• Filed: 08/20/1998
• Issued: 09/11/2001
• Est. Priority Date: 11/02/1994
• Status: Expired due to Term

• Current Assignee: Metricly, Inc. (Virtana Corp.)
• Original Assignee: Netuitive, Inc. (Virtana Corp.)
• Inventors: Jannarone, Robert
• Primary Examiner(s): Powell, Mark P.
• Assistant Examiner(s): Booker, Kelvin
• Application Number:
  US09/137,265
• Time in Patent Office: 1,118 Days
• Field of Search: 706/25, 706/26, 706/27
• US Class Current: 706/26
• CPC Class Codes: G06F 18/2113 by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/10 Interfaces, programming languages or software development kits, e.g. for simulating neural networks
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space

Abstract

The present invention provides a system for learning from and responding to regularly arriving information at once by quickly combining prior information with concurrent trial information to produce useful learned information. A parallel embodiment of the system performs can perform updating operations for memory elements of matrix through the coordinated use of parallel feature processors and a joint access memory, which contains weighted values and provision for connecting feature processors pairwise. The parallel version also performs feature function monitoring, interpretation and refinement operations promptly and in concert with concurrent operation. A non-parallel embodiment of the system uses a single processor to perform the above operations however, more slowly than the parallel embodiment, yet faster than available alternatives.

Claims

1. A parallel processing system for computing output values from input values received during a time trial, comprising:

a plurality of processing units, each of said processing units operative to receive, during a time trial and in parallel, individual input values from an input vector, said processing units comprising analogs components for computing arithmetic operations of said system;

a plurality of interconnected conductors, operative to connect each of said processing units to every other processing unit of said system and operative to transfer weighted values among said processors;

a plurality of switching junctions located along said interconnected conductors, said switching junctions being operable for uniquely pairing each of said processors to every other processor of said system; and

each of said processing units operative to provide, during said time trial, an expected output value based on said weighted values and each of said processing units operative to update, during said time trial, said weighted values based on said input values.

2. The system of claim 1 wherein analog components are operative to calculate values for updating said connection weights to transmitting to other processors of said system.

3. The system of claim 1 wherein said switching junctions comprise analog components for computing connection weight at said switching junctions.

4. The system of claim 1 wherein said switching junction have memory elements.

5. The system of claim 4 wherein said memory elements are resistive-capacitance elements.

6. The system of claim 1 wherein each of said conductors is a single wire conductor.

7. The system of claim 1 wherein said analog components receive input analog signals and produce an analog output having a magnitude corresponding to the magnitude of the inputs as operated on by the function associated with the analog component.

8. The apparatus of claim 3 further comprising a plurality of memory elements, each of said memory elements being individually coupled to a separate switching junction and each of said memory elements containing a weighted memory value.

9. A computer processing system for updating a matrix of elements for use in determining the relationship among a plurality of variables, comprising:

a plurality of processors, each of said processors operative to receive an input variable of a plurality of input variables in parallel received during a time trial;

an ordered array of memory elements associated with each processor, each of said memory elements defining elements of a matrix, each of said memory elements containing values that are operative to define a relationship among received input variables and each of said memory elements operative to be updated by each of said processors;

each of said processors operative to be uniquely paired with every other processor of said computer system;

said uniquely paired processors sharing data via a memory element associated with each unique pair of processors; and

each of said processors updating said memory elements to reflect the relationship among input variables received during a time trial.

10. The system of claim 9 wherein said memory elements contain values that represent a matrix inverse.

11. The system of claim 10 wherein said matrix inverse is an inverse of a matrix of second order derivatives.

12. The system of claim 11 wherein said matrix of elements are updated according to the equation:

13. The system of claim 12 wherein the inner product C[DFP]x[DFP]^Tx[DFP] is computed in a first step;

and the inner product b[DFP]y[DFP]^Ty[DFP] is computed in a second step.

14. The system of claim 9 wherein said matrix of elements are updated according to the equation:

15. The system of claim 9 wherein said matrix is a connection weight matrix used for predicting outputs based on input variables received.

16. The system of claim 9 further comprising a plurality of switching junctions for uniquely pairing each of said processors to every other processor of said system.

17. The apparatus of claim 16 wherein each of said switching junctions selectively connects each of said processors to only one other of said processors during a time interval, thereby forming multiple paired sets of said processors, for communicating said weighted values during a time interval.

18. The apparatus of claim 17 wherein each of said memory elements is individually coupled to a separate switching junction.

19. The apparatus of claim 17 wherein said switching junctions successively connect different sets of multiple paired sets of said processors during multiple time intervals.

20. The system of claim 19 wherein said switching junctions successively connect all possible combinations of said different sets of multiple paired sets of said processors in a minimum number of steps.

21. A computer processing system for computing a matrix of elements, comprising:

a plurality of processing units, each of said processing units operative to receive, during a time trial and in parallel, individual input values from an input vector e[IN];

an ordered array of memory elements associated with each processor, each of said memory elements defining elements of a symmetric matrix ω

;

each of said processors operative to be uniquely paired with every other processor of said computer system, thereby forming a processor pair ƒ and

g;

said uniquely paired processors sharing data via a memory element uniquely associated with each unique pair of processors; and

each of said processors computing said memory elements during a time trial by multiplying said input vector e[IN] by said symmetric matrix ω

.

22. The system of claim 21 further comprising a plurality of switching junctions for uniquely pairing each of said processors to every other processor of said system.

23. The apparatus of claim 22 wherein each of said switching junctions selectively connects each of said processors to only one other of said processors during a time interval, thereby forming multiple paired sets of said processors, for communicating said weighted values during a time interval.

24. The apparatus of claim 22 wherein each of said memory elements is individually coupled to a separate switching junction.

25. The apparatus of claim 22 wherein said switching junctions successively connect different sets of multiple paired sets of said processors during multiple time intervals.

26. The system of claim 25 wherein said switching junctions successively connect all possible combinations of said different sets of multiple paired sets of said processors in a minimum number of steps.

27. A computer processing system for modifying the elements ω

{ƒ

,ƒ

} of a connection weight matrix used to produce an output based on inputs received during a time trial, comprising;

a plurality of processing units, each of said processing units operative to receive, during a time trial and in parallel, individual input variables from an input vector ω

<

ƒ

>

;

an ordered array of memory elements associated with each processor, each of said memory elements defining elements of a matrix ω

<

ƒ

,ƒ

>

, each of said memory elements containing values that are operative to define a relationship among received input variables and each of said memory elements operative to be modified by each of said processors;

each of said processors operative to be uniquely paired with every other processor of said computer system;

said uniquely paired processors sharing data via a memory element uniquely associated with each unique pair of processors; and

each of said processors updating said memory elements to reflect the relationship among input variables received during a time trial.

28. The system of claim 27 wherein said matrix ω

{ƒ

,ƒ

} is updated in parallel by said processors according to the following process ω

{ƒ

,ƒ

}=ω

<

ƒ

,ƒ

>

−

ω

<

ω

>

^Tω

<

ω

>

/ω

(ƒ

,ƒ

), where ω

<

ƒ

>

represents an input vector, ƒ

<

ƒ

,ƒ

>

is a symmetric matrix of memory elements, and ω

(ƒ

,ƒ

) is a constant.

29. The system of claim 28 wherein said matrix of element is the inverse of a matrix.

30. The system of claim 27 further comprising a plurality of switching junctions for uniquely pairing each of said processors to every other processor of said system.

31. The apparatus of claim 30 wherein each of said switching junctions selectively connects each of said processors to only one other of said processors during a time interval, thereby forming multiple paired sets of said processors, for communicating weighted values during, a time interval.

32. The apparatus of claim 30 wherein each of said memory elements is individually coupled to a separate switching junction.

33. The apparatus of claim 30 wherein said switching junctions successively connect different sets of multiple paired sets of said processors during multiple time intervals.

34. The system of claim 33 wherein said switching junctions successively connect all possible combinations of said different sets of multiple paired sets of said processors in a minimum number of steps.

35. A computer processing system for updating a matrix of elements for use in determining the relationship among a plurality of variables, comprising:

a plurality of processors, each of said processing units operative to receive, during a time trial and in parallel, individual input values from an input vector m[IN];

a plurality of interconnected conductors, operative to connect each of said processing units to every other processing unit of said system and operative to transfer weighted values among said processors;

a variance-covariance matrix of elements ν

associated with a processor, each of said memory elements being variance-covariance matrix values ν

defining elements of a matrix, each of said memory elements containing values that are operative to define a relationship among received input variables and each of said memory elements operative to be updated, by each of said processors, according to the equation;

36. The system of claim 35 wherein each of said processors is operative to be uniquely paired with every other processor of said computer system;

and

said uniquely paired processors sharing data via a memory element uniquely associated with each unique pair of processors.

37. The system of claim 36 further comprising a plurality of switching junctions for uniquely pairing each of said processors to every other processor of said system.

38. The apparatus of claim 37 further comprising each of said switching junctions selectively connects each of said processors to only one other of said processors during a time interval, thereby forming multiple paired sets of said processors, for communicating said weighted values during a time interval.

39. The apparatus of claim 37 wherein each of said memory elements is individually coupled to a separate switching junction.

40. The apparatus of claim 37 wherein said switching junctions successively connect different sets of multiple paired sets of said processors during multiple time intervals.

41. The system of claim 40 wherein said switching junctions successively connect all possible combinations of said different sets of multiple paired sets of said processors in a minimum number of steps.

42. A method for updating a matrix of elements for use in determining the relationship among a plurality of variables in a computer processing system, comprising:

receiving individual input values from an input vector m[IN]; and

updating a variance-covariance matrix of elements ν

associated with a processor according to the equation;

43. The method of claim 42 further comprising inverting said matrix of elements for use in providing output values from said input values.

44. A processing system for computing output values from input values received during a series of time trials, comprising:

a processing unit operative to receive an input vector comprising a plurality of input data values for each time trial;

a memory unit, connected to the processing unit, containing elements defining a connection weight matrix that is the inverse of a covariance matrix based on regression analysis applied to input data values received during prior time trials;

said processing unit operative to provide, during each time trial, one or more output values calculated as a function of the input data values for the time trial and the elements of the connection weight matrix; and

said processing unit operative to update the elements of the connection weight matrix, during each time trial, based on covariance relationships among the input values received during the time trial.

45. The processing system of claim 44, wherein the output values for each time trial correspond to a weighted sum of the input data values for the time trial, in which the weights are defined by the elements of the connection weight matrix.

46. The processing system of claim 44, wherein the processing unit is operative to access each element of the connection weight matrix in a sequential order.

47. The processing system of claim 44, wherein the processing unit is operative to access the elements of the connection weight matrix in parallel.

48. The processing system of claim 44, wherein the processing unit is operative to impute output values corresponding to missing input data values, during each time trial, based on the input data values for the time trial and the elements of the connection weight matrix.

49. The processing system of claim 44, wherein the processing unit is operative to impute output values corresponding to deviant input data values, during each time trial, based on the input data values for the time trial and the elements of the connection weight matrix.

50. The processing system of claim 44, wherein the input values correspond to measurements, and the output values correspond to expected values for the measurements.

51. The processing system of claim 44, wherein the processing unit updates each element of the connection weight matrix, for each time trial, using a learning weight reflecting a weight given to the input values for the time trial relative to weights given to corresponding input values for previous time trials.

52. The processing system of claim 51, further comprising a learning weight controller operative for changing the learning weight for each element of the connection weight matrix from time trial to time trial.

53. The processing system of claim 52, wherein the learning weight controller is operative for disabling learning in response to detecting abnormal deviation of input values.

54. The processing system of claim 52, wherein the learning weight controller is operative for receiving user inputs defining one or more of the learning weights.

55. The processing system of claim 44, further comprising a feature function controller for receiving measurements and computing the input vector based on the measurements.

56. The processing system of claim 55, wherein the feature function controller is operative for receiving user inputs defining values for the elements of the connection weight matrix.

57. The processing system of claim 56, wherein the connection weight matrix is initialized through the user defined values for the elements of the connection weight matrix.

58. The processing system of claim 55, wherein the feature function controller is operative for receiving output feature values from the processing unit and computing the output values based on the output feature values.

59. The processing system of claim 55, wherein the feature function controller is operative for computing statistics to identify sudden changes in input values.

60. The processing system of claim 55, wherein the feature function controller is operative for computing statistics to identify sudden changes in output values.