Method and apparatus for determining the sensitivity of inputs to a neural network on output parameters

• US 5,825,646 A
• Filed: 06/03/1996
• Issued: 10/20/1998
• Est. Priority Date: 03/02/1993
• Status: Expired due to Term

• Current Assignee: Rockwell Automation Technologies Incorporated (Rockwell Automation, Inc.)
• Original Assignee: Pavilion Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Liano, Kadir, Hartman, Eric J., Keeler, James David
• Primary Examiner(s): Lee, Thomas C.
• Assistant Examiner(s): Garland, Steven R.
• Application Number:
  US08/656,702
• Time in Patent Office: 869 Days
• Field of Search: 364/148-153, 364/164, 364/165, 395/22, 395/23, 395/75
• US Class Current: 700/44
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: G05B 13/0285 using neural networks and fuzzy logic
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A distributed control system (14) receives on the input thereof the control inputs and then outputs control signals to a plant (10) for the operation thereof. The measured variables of the plant and the control inputs are input to a predictive model (34) that operates in conjunction with an inverse model (36) to generate predicted control inputs. The predicted control inputs are processed through a filter (46) to apply hard constraints and sensitivity modifiers, the values of which are received from a control parameter block (22). During operation, the sensitivity of output variables on various input variables is determined. This information can be displayed and then the user allowed to select which of the input variables constitute the most sensitive input variables. These can then be utilized with a control network (470) to modify the predicted values of the input variables. Additionally, a neural network (406) can be trained on only the selected input variables that are determined to be the most sensitive. In this operation, the network is first configured and trained with all input nodes and with all training data. This provides a learned representation of the output wherein the combined effects of all other input variables are taken into account in the determination of the effect of each of the input variables thereon. The network (406) is then reconfigured with only the selected inputs and then the network (406) again trained on only the input/output pairs associated with the select input variables.

Claims

1. A system for determining a set of sensitive inputs in a non-linear predictive network having stored therein a representation of a plant, comprising:

a configurable network having;

an input layer for receiving a defined number of input variables, which defined number of input variables is a variable number,

an output layer for outputting a defined number of outputs, each associated with an output variable, and

a hidden layer for mapping said input layer to said output layer through a stored learned representation of the plant;

a dataset of training patterns representing substantially all of the input variables to the plant and associated measured output variables of the plant, with each of said training patterns having input values associated with said input variables and corresponding output values associated with said output variables;

a training system for training said configurable network in accordance with a predetermined training algorithm;

a sensitivity processor for determining as a sensitivity value the effect of each of said input variables input to said configurable network on a learned representation of said output variables, wherein the combined effects of all other input variables are taken into account;

a central processor for configuring said network to receive substantially all of said input variables in a first mode and controlling said training system to train said configurable network with substantially all of said input variables and said output variables in said dataset;

said central processor controlling said sensitivity processor to determine said sensitivity values associated with each of said input variables and compare said determined sensitivity values with a predetermined threshold and select only the ones of said input variables from said dataset having a sensitivity value that exceeds said sensitivity threshold, this being defined as a selected dataset; and

said central processor operable to configure said network to receive only input variables from said received input variables from said selected dataset and control said training system to train said configurable network on said selected dataset to provide a stored learned representation of a plant trained only on the selected input variables.

2. The system of claim 1 wherein said training algorithm comprises a backpropagation training algorithm and said non-linear predictive network comprises a neural network.

3. A control network for predicting control inputs for input to a distributed control system, which distributed control system outputs controls for input to a plant, the control inputs associated with the input variables to the plant, comprising:

a control model for storing a representation of the plant;

a desired value input defining desired operating parameters for the plant;

said control model operable to receive as inputs the control inputs output by the distributed control system and said desired value input, and predicted control input values for input as updated control inputs to the distributive distributed control system that are necessary to achieve said desired operating parameters associated with said desired value input;

a memory for storing sensitivity modifiers for defining the effect of each of the input variables on the plant output; and

a modifier circuit for modifying the value of the predicted control input values prior to input to the distributed control system in accordance with a predetermined function of said pre-stored sensitivity modifiers in said memory.

4. The control network of claim 3 wherein said control model comprises a neural network having an input layer and an output layer with a hidden layer for mapping said input layer to said output layer through said stored representation of the plant.

5. The control network of claim 3 wherein said control model comprises:

a predictive network having stored therein a representation of the plant and for receiving the control inputs output by the distributed control system and generating a predicted plant output;

an error generator for determining the error between said desired value input and the predicted output of said predictive network; and

an inverse predictive network substantially similar to said predictive network for receiving said error value on the output thereof and propagating the error through the network in an inverse direction to the input thereof and changing the input values to minimize said error, the input of said inverse network outputting said predicted control inputs.

6. The control network of claim 5 wherein said error is propagated through said inverse predictive network and the error minimized in accordance with a backpropagation-to-activation algorithm.

7. The control network of claim 3 and further comprising an analysis system for determining said sensitivity modifiers and, after determination thereof, storing said sensitivity modifiers in said memory.

8. The control network of claim 7 wherein said stored representation is a learned representation and said analysis system comprises:

a dataset memory for storing a plurality of training patterns on which said control model was trained to learn said stored representation, said training patterns representing the input variables and measured output variables available to train said control model on in order to generate said learned representation, with each of said training patterns having input values associated with said input variables and corresponding output values associated with said associated output variables; and

means for determining the effect of each of the input variables on a learned representation of said output variables, which learned representation of said output variables was learned as a function of all of the input variables, and wherein the combined effects of all input variables are taken into account in said learned representation of said output variables.

9. The control network of claim 8 wherein there are I output variables o_i and j input variables x_j, wherein said means for determining is operable to calculate said sensitivity modifiers for the jth variable/ith output variable, combination as follows:

##EQU19## where N_pats is the number of training patterns in said memory and o_k,i is the ith output for the kth pattern.

10. The control network of claim 8 wherein there are I output variables o_i and j input variables x_j, wherein said means for determining is operable to calculate said sensitivity modifiers for the jth variable/ith output variable, combination as follows:

##EQU20## where N_pats is the number of training patterns in said memory and _ok,i is the ith output for the kth pattern.

11. The control network of claim 8 wherein there are I output variables o_i and j input variables x_j, wherein said means for determining is operable to calculate said sensitivity modifiers for the jth variable/ith output variable, combination as follows:

##EQU21## where N_pats is the number of training patterns in said memory and o_k,i is the ith output for the kth pattern.

12. A method for generating control inputs with a distributed control system that generates control inputs for a plant, comprising the steps of:

providing a control model that contains a representation of a plant;

inputting a desired input value that defines desired operating parameters for the plant;

inputting the control inputs generated by the distributed control system to the control model;

predicting control input values that will allow the plant to operate at the desired input value by processing the desired input value and current control inputs from the distributed control system to generate an error and minimizing the error with the control model, the predicted control inputs operable to be input to the distributed control system in order for the distributed control system generate new control inputs;

storing sensitivity modifiers for defining the effect of each of the input variables on the plant output; and

modifying the value of the predicted control input values prior to input to the distributed control system in accordance with a predetermined function of the stored sensitivity modifiers.

13. The method of claim 12 wherein the step of providing the control model comprises providing a neural network having an input layer and an output layer with a hidden layer for mapping the input layer to the output layer through the stored representation of the plant.

14. The method of claim 12 wherein the step of providing the control model comprises:

providing a predictive network having stored therein a representation of the plant and for receiving the control inputs output by the distributed control system and predicting a plant output;

providing an inverse predictive network substantially similar to the predictive network for receiving an error value on the output thereof and propagating the error through the inverse predictive network in an inverse direction to the input and changing the input values to minimize the error;

the step of predicting the control inputs comprising;

comparing the predicted plant output from the predictive network to the desired input value and generating the error for input to the output of the inverse predictive network, and

controlling the predictive network and inverse network to minimize the error in the step of determining the error by alternately processing the input values from the inverse predictive network through the predictive network to generate a new error and then reprocess the error through the inverse network until the error is minimized, the value output from the input of the inverse predictive network comprising the predicted control inputs.

15. The method of claim 14 wherein the error is propagated through the inverse predictive network and the error minimized in accordance with a backpropagation-to-activation algorithm.

16. The method of claim 12 and further comprising determining the sensitivity modifiers and, after termination thereof, storing the sensitivities in the step of storing.

17. The method of claim 16 wherein the stored representation of the control model is a learned representation and the step of determining the sensitivity modifiers comprises:

storing a plurality of training patterns in a dataset memory, with the training patterns comprising patterns on which the control model was trained to learn the stored representation, the training patterns representing the input variables and measured output variables available to train the control model in order to generate the learned representation, with each of the training patterns having input values associated with the input variables and corresponding output values associated with the associated output variables; and

determining the effect of each of the input variables on a learned representation on the output variables, which learned representation of the output variables was learned as a function of all of the input variables, and wherein the combined effects of all input variables are taken into account in the learned representation of the output variables.

18. The method of claim 17 wherein there are I output variables o_i and j input variables x_j, wherein the step of determining is operable to calculate the sensitivity modifiers for the jth variable/ith output variable combination as follows:

##EQU22## where N_pats is the number of training patterns in the dataset memory and o_k,i is the ith output for the kth pattern.

19. The method of claim 17 wherein there are I output variables o_i and j input variables x_j, wherein step of determining is operable to calculate the sensitivity modifiers for the jth variable/ith output variable combination as follows:

##EQU23## where N_pats is the number of training patterns in the dataset memory and o_k,i is the ith output for the kth pattern.

20. The method of claim 17 wherein there are I output variables o_i and j input variables x_j, wherein step of determining is operable to calculate the sensitivity modifiers for the jth variable/ith output variable combination as follows:

##EQU24## where N_pats is the number of training patterns in the dataset and o_k,i is the ith output for the kth pattern.