Multi-kernel neural network concurrent learning, monitoring, and forecasting system

• US 6,216,119 B1
• Filed: 11/19/1997
• Issued: 04/10/2001
• Est. Priority Date: 11/19/1997
• Status: Expired due to Term

• Current Assignee: Metricly, Inc. (Virtana Corp.)
• Original Assignee: Netuitive, Inc. (Virtana Corp.)
• Inventors: Jannarone, Robert J.
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): Starks, Wilbert L.
• Application Number:
  US08/974,377
• Time in Patent Office: 1,238 Days
• Field of Search: 382/103, 382/104, 382/232, 382/219
• US Class Current: 706/26
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

A multi-kernel neural network computing architecture configured to learn correlations among feature values 34, 38 as the network monitors and imputes measured input values 30 and also predicts future output values 46. This computing architecture, referred to as a concurrent-learning information processor (CIP 10), includes a multi-kernel neural network array 14 with the capability to learn and predict in real time. The CIP 10 also includes a manager 16 and an input-output transducer 12 that may be used for input-output refinement. These components allow the computing capacity of the multi-kernel array 14 to be reassigned in response to measured performance or other factors. The output feature values 46 computed by the multi-kernel array 14 and processed by an output processor 44 of the transducer 12 are supplied to a response unit 18 that may be configured to perform a variety of monitoring, forecasting, and control operations in response to the computed output values. Important characteristics of the CIP 10, such as feature function specifications 35 and 49, connection specifications 42, learning weight schedules 55, and the like may be set by a technician through a graphical user interface 20. Refinement processes also allow the CIP 10 be reconfigured in accordance with user commands for application to different physical applications.

Claims

1. A method for responding to computed output values that are based on measured input values received during a current time trial and during one or more historical time trials, comprising the steps of:

(a) receiving an iteration of the measured input values for the current time trial;

(b) assembling a vector of input feature values based on the measured input values;

(c) providing the vector of input feature values to a multi-kernel processor, each kernel of the processor operative for;

receiving one or more of the input feature values,

retrieving connection specifications defining mathematical relationships for computing one or more output feature values based on the received input feature values,

retrieving a set of connection weights reprinting regression coefficients among the received input feature values and the computed output feature values,

retrieving a set of learning weights defining mathematical relationships for updating the connection weights based on the received input feature values,

computing the output feature values based on the received input feature values, the connection weights, and the connection specifications,

computing updated connection weights based on the received input feature values, the connection weights, the connection specifications, and the leaning weights,

storing the updated connection weights, and

providing access to the computed output feature values;

(d) assembling a vector of computed output values based on the output feature values computed by each kernel;

(e) responding to the vector of computed output values;

(f) determining whether refinement operations are indicated, and if refinement operations are not indicated, repeating steps (a) through (f) for a subsequent time trial; and

(e) if refinement operations are indicated, performing one or more refinement operations selected from the group including;

recomputing the learning weights based on the measured input values and the computed output values for a plurality of time trials;

recomputing the connection specifications based on the measured input values and the computed output values for a plurality of time trials;

deleting ineffective input or output feature values;

combining redundant input or output feature values;

specifying new input or output feature values;

recomputing the input feature specifications based on the measured input values and the computed output values for a plurality of time trials;

recomputing the output feature specifications based on the measured input values and the computed output values for a plurality of time trials; and

reassigning functionality among the kernels.

2. The method of claim 1, wherein:

the connection weights comprise the elements of an inverse covariance matrix; and

the step of computing the updated connection weights comprises the step of updating the inverse covariance matrix.

3. The method of claim 1, wherein:

the connection weights comprise the elements of an inverse covariance matrix; and

the step of computing updated connection weights comprises the steps of,

updating a covariance matrix corresponding to the inverse covariance matrix, and

inverting the updated covariance matrix.

4. The method of claim 1, wherein the step of assembling the vector of input feature values comprises the step of computing the input feature values based on the measured input values and input feature specifications.

5. The method of claim 4, wherein the step of computing the input feature values comprises a mathematical operation selected from the group including:

computing input feature values based on algebraic combinations of measured input values;

computing input feature values based on coefficients corresponding to a polynomial approximating a function defined by measured input values;

computing input feature values based on coefficients corresponding to a differential equation corresponding to a function defined by measured input values; and

computing input feature values based on coefficients corresponding to a frequency-domain function corresponding to a function defined by measured input values.

6. The method of claim 5, wherein the step of assembling the vector of computed output values comprises the step of performing a mathematical inverse of one of the mathematical operations performed on the measured input values.

7. The method of claim 4, wherein the step of assembling the vector of computed output values comprises the step of computing the output values based on the output feature values and output feature specifications.

8. The method of claim 7, wherein the step of computing the output feature values comprises the steps of:

imputing output feature values for the current time trial based on the input feature values for one or more historical time trials;

computing monitored output feature values based on the input feature values for the current time trial;

computing deviance values based on the imputed output feature values and the monitored output feature values; and

basing the computed output values on the monitored output feature values.

9. The method of claim 8, wherein the step of responding to the vector of computed output values comprises the steps of:

comparing each deviance value to an associated threshold value; and

if one of the deviance values exceeds its associated threshold value, performing one or more deviance operations selected from the group including,

indicating an alarm condition, and

basing the computed output values on the imputed output feature value rather than the monitored output feature value for the output feature value associated with the deviance value that exceeds its associated threshold value.

10. The method of claim 7, if refinement operations are indicated, further comprising the step of:

recomputing the learning weights based on the measured input values and the computed output values for a plurality of time trials; and

recomputing the connection specifications based on the measured input values and the computed output values for a plurality of time trials.

11. The method of claim 1, wherein the step of computing the output feature values comprises the step of predicting output feature values for future time trials.

12. The method of claim 1, wherein the step of computing the output feature values comprises the steps of:

imputing output feature values for the current time trial based on the input feature values for one or more historical time trials; and

predicting output feature values for future time trials.

13. The method of claim 1, wherein the step of responding to the vector of computed output values comprises the step of performing one or more control operations selected from the group including,

displaying a representation of the computed output values on a display device, and

actuating a controlled parameter to compensate for a condition indicated by the computed output values.

14. The method of claim 1, wherein the multi-kernel processor comprises:

an array of spatially-dedicated kernels corresponding to a spatially-contiguous field from which input values are measured and for which output valued are predicted; and

each kernel configured to compute one of the computed output values based on a set of adjacent measured input values.

15. The method of claim 14, wherein:

each kernel of the multi-kernel processor corresponds to a mutually-exclusive time-specific price forecast for a commodity price index; and

each kernel configured to predict its corresponding mutually-exclusive time-specific price forecast based on the measured input values.

16. The method of claim 15, wherein the measured input values comprise:

the commodity price index; and

price indices for currencies and other commodities.

17. The method of claim 1, wherein the multi-kernel processor comprises:

an array of temporally-dedicated kernels corresponding to a time-based index from which input values are measured and for which output values are predicted; and

each kernel configured to predict a mutually-exclusive one of the time-based index values based on the measured input values.

18. The method of claim 1, wherein the multi-kernel processor comprises:

a plurality of kernel groups each comprising a plurality of temporally-dedicated kernels corresponding to a time-based index from which input values are measured and for which output valued are predicted;

each kernel group comprising a plurality of individual kernels, each configured to predict a component of a mutually-exclusive time-based index value based on the measured input values;

the plurality of groups of temporarily-dedicated kernels defining an array of spatially-dedicated kernel groups;

each kernel group configured to compute a component of the time-based index.

19. The method of claim 1, wherein:

the time-based index comprises an electricity demand index;

each kernel group corresponds to a plurality of electricity delivery points; and

the measured input values comprise electricity demand and weather data.

20. The method of claim 1, wherein:

each kernel of the multi-kernel processor corresponds to a pixel in a visual image;

each measured input value corresponds to a measured intensity of one of the pixels of the visual image; and

each computed output value corresponds to a computed intensity of one of the pixels of the visual image.

21. A method for responding to computed output values that are based on measured input values received during a current time trial and during one or more historical time trials, comprising the steps of:

receiving an iteration of the measured input values for the current time trial;

assembling a vector of input feature values based on the measured input values;

providing the vector of input feature values to a multi-kernel processor comprising a monitoring sub-array and a forecasting sub-array, each kernel of the processor operative for,

receiving one or more of the input feature values,

retrieving connection specifications defining mathematical relationships for computing one or more output feature values based on the received input feature values,

retrieving a set of connection weights representing regression coefficients among the received input feature values and the computed output feature values,

retrieving a set of learning weights defining mathematical relationships for updating the connection weights based on the received input feature values,

imputing output feature values for the monitoring sub-array based on the input feature values for one or more historical time trials, the connection weights, and the connection specifications,

computing updated connection weights for the forecasting sub-array based on the received input feature values for the current time trial, the connection weights, the connection specifications, the learning weights, and the imputed output feature values,

computing output feature values for the forecasting sub-array based on the input feature values for one or more historical time trials, the updated connection weights, and the connection specifications,

providing access to the output feature values,

computing updated connection weights for the monitoring sub-array based on the received input feature values, the connection weights, the connection specifications, and the learning weights, and

storing the updated connection weights for the monitoring sub-array and for the forecasting sub-array;

assembling a vector of computed output values based on the output feature values computed by each kernel; and

responding to the vector of computed output values.

22. The method of claim 21, wherein:

the connection weights for the monitoring sub-array comprise the elements of an inverse covariance matrix for the monitoring sub-array; and

the step of computing the updated connection weights for the monitoring sub-array comprises the step of updating the inverse covariance matrix for the monitoring sub-array;

the connection weights for the forecasting sub-array comprise the elements of an inverse covariance matrix for the forecasting sub-array; and

the step of computing the updated connection weights for the forecasting sub-array comprises the step of updating the inverse covariance matrix for the forecasting sub-array.

23. The method of claim 21, wherein:

the connection weights for the monitoring sub-array comprise the elements of an inverse covariance matrix for the monitoring sub-array;

the step of computing updated connection weights for the monitoring sub-array comprises the steps of,

updating a covariance matrix for the monitoring sub-array corresponding to the inverse covariance matrix for the monitoring sub-array, and

inverting the updated covariance matrix for the monitoring sub-array;

the connection weights for the forecasting sub-array comprise the elements of an inverse covariance matrix for the forecasting sub-array; and

the step of computing updated connection weights for the forecasting sub-array comprises the steps of,

updating a covariance matrix for the forecasting sub-array corresponding to the inverse covariance matrix for the forecasting sub-array, and

inverting the updated covariance matrix for the forecasting sub-array.

24. A computer-readable medium having computer-executable instructions for performing the steps of responding to computed output values that are based on measured input values received during a current time trial and during one or more historical time trials, comprising the steps of:

(a) receiving an iteration of the measured input values for the current time trial;

(b) assembling a vector of input feature values based on the measured input values;

(c) providing the vector of input feature values to a multi-kernel processor, each kernel of the processor operative for;

receiving one or more of the input feature values,

retrieving connection specifications defining mathematical relationships for computing one or more output feature values based on the received input feature values,

retrieving a set of connection weights representing regression coefficients among the received input feature values and the computed output feature values,

retrieving a set of learning weights defining mathematical relationships for updating the connection weights based on the received input feature values,

computing the output feature values based on the received input feature values, the connection weights, and the connection specifications,

computing updated connection weights based on the received input feature values, the connection weights, the connection specifications, and the learning weights,

storing the updated connection weights, and

providing access to the computed output feature values;

(d) assembling a vector of computed output values based on the output feature values computed by each kernel;

(e) responding to the vector of computed output values; and

(f) determining whether refinement operations are indicated, and if refinement operations are not indicated, repeating steps (a) through (e) for a subsequent time trial.

25. The method of claim 24, wherein:

the connection weights comprise the elements of an inverse covariance matrix; and

the step of computing the updated connection weights comprises the step of updating the inverse covariance matrix.

26. The method of claim 24, wherein:

the connection weights comprise the elements of an inverse covariance matrix; and

the step of computing updated connection weights comprises the steps of,

updating a covariance matrix corresponding to the inverse covariance matrix, and

inverting the updated covariance matrix.

27. The computer-readable of claim 24, wherein the step of computing the output feature values comprises the steps of:

imputing output feature values for the current time trial based on the input feature values for one or more historical time trials; and

predicting output feature values for future time trials.

28. The computer-readable medium of claim 27, wherein the step of responding to the vector of computed output values comprises the steps of:

comparing each deviance value to an associated threshold value; and

if one of the deviance values exceeds its associated threshold value, performing one or more deviance operations selected from the group including,

indicating an alarm condition, and

basing the computed output values on the imputed output feature value rather than the monitored output feature value for the output feature value associated with the deviance value that exceeds its associated threshold value.

29. The computer-readable medium of claim 27, wherein the step of responding to the vector of computed output values comprises the step of performing one or more control operations selected from the group including,

displaying a representation of the computed output values on a display device, and

actuating a controlled parameter to compensate for a condition indicated by the computed output values.

30. The computer-readable medium of claim 27, further comprising the step of performing one or more refinement operations selected from the group including:

deleting ineffective input or output feature values;

combining redundant input or output feature values;

specifying new input or output feature values;

recomputing the input feature specifications based on the measured input values and the computed output values for a plurality of time trials;

recomputing the learning weights based on the measured input values and the computed output values for a plurality of time trials;

recomputing the connection specifications based on the measured input values and the computed output values for a plurality of time trials;

recomputing the output feature specifications based on the measured input values and the computed output values for a plurality of time trials; and

reassigning functionality among the kernels.

31. The computer-readable medium of claim 27, further comprising the step of:

recomputing the learning weights based on the measured input values and the computed output values for a plurality of time trials; and

recomputing the feature value specifications based on the measured input values and the computed output values for a plurality of time trials.

32. The computer-readable medium of claim 24, wherein the step of computing the output feature values comprises the steps of:

imputing output feature values for the current time trial based on the input feature values for one or more historical time trials;

computing monitored output feature values based on the input feature values for the current time trial;

computing deviance values based on the imputed output feature values and the monitored output feature values; and

basing the computed output values on the monitored output feature values.

33. A computer system comprising:

an input processor configured for,

receiving an iteration of the measured input values for the current time trial, and

assembling a vector of input feature values based on the measured input values;

a multi-kernel processor coupled to the input processor and configured for receiving the vector of input feature values, each kernel of the processor operative for

receiving one or more of the input feature values,

retrieving connection specifications defining mathematical relationships for computing one or more output feature values based on the received input feature values,

retrieving a set of connection weights representing regression coefficients among the received input feature values and the computed output feature values,

retrieving a set of learning weights defining mathematical relationships for updating the connection weights based on the received input feature values,

computing the output feature values based on the received input feature values, the connection weights, and the connection specifications,

computing updated connection weights based on the received input feature values, the connection weights, the connection specifications, and the learning weights,

storing the updated connection weights, and

providing access to the computed output feature values;

an output processor coupled to the multi-kernel array and configured for assembling a vector of computed output values based on the output feature values computed by each kernel;

a response unit coupled to the output processor and configured for responding to the vector of computed output values; and

a manager coupled to the multi-kernel array and operative for;

recomputing the learning weighs based on the measured input values and the computed output values for a plurality of time trials; and

recomputing the connection specifications based on the measured input values and the computed output values for a plurality of time trials.

34. The computer system of claim 33, wherein:

the connection weights associated with each kernel comprise the elements of an inverse covariance matrix; and

each kernel is operative for computing the updated connection weights by updating the inverse covariance matrix.

35. The computer system of claim 33, wherein:

the connection weights associated with each kernel comprise the elements of an inverse covariance matrix; and

each kernel is operative for,

updating a covariance matrix corresponding to the inverse covariance matrix, and

inverting the updated covariance matrix.

36. The computer system of claim 35, wherein the response unit is configured for:

displaying a representation of the computed output values on a display device,

indicating an alarm condition, or

actuating a controlled parameter to compensate for a condition indicated by the computed output values.

37. The computer system of claim 33, wherein the manager is coupled to the input processor and the output processor, and the manager is further operative for:

deleting ineffective input or output feature values;

combining redundant input or output feature values;

specifying a new input or output feature values;

recomputing input or output feature specifications based on the measured input values and the computed output values for a plurality of time trials; and

reassigning functionality among the kernels.

38. The computer system of claim 33, further comprising a learned parameter memory coupled to the multi-kernel array and operative for receiving the connection weights from the multi-kernel array, storing the connection weights, and providing the connection weights to the multi-kernel array.