Method and apparatus for training a system model with gain constraints

• US 20060074501A1
• Filed: 11/04/2005
• Published: 04/06/2006
• Est. Priority Date: 05/06/1996
• Status: Active Grant

• Current Assignee: Rockwell Automation Technologies Incorporated (Rockwell Automation, Inc.)
• Original Assignee: Pavilion Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Piche, Stephen, Hartman, Eric Jon, Gerules, Mark
• Granted Patent: US 7,213,006 B2
• Time in Patent Office: Days
• US Class Current: 700/29
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: G05B 13/042 in which a parameter or coefficient is automatically adjusted to optimise the performance
• CPC Class Codes: G05B 13/048 using a predictor
• CPC Class Codes: G05B 17/02 electric
• CPC Class Codes: G06N 3/02 Neural networks

Abstract

Method and apparatus for training a system model with gain constraints. A method is disclosed for training a steady-state model, the model having an input and an output and a mapping layer for mapping the input to the output through a stored representation of a system. A training data set is provided having a set of input data u(t) and target output data y(t) representative of the operation of a system. The model is trained with a predetermined training algorithm which is constrained to maintain the sensitivity of the output with respect to the input substantially within user defined constraint bounds by iteratively minimizing an objective function as a function of a data objective and a constraint objective. The data objective has a data fitting learning rate and the constraint objective has constraint learning rate that are varied as a function of the values of the data objective and the constraint objective after selective iterative steps.

Claims

1. A system, comprising:

a model, comprising;

a linear portion; and

a non-linear portion;

wherein the model comprises a representation of a plant or process;

wherein the model is operable to receive an input vector comprising one or more inputs, and compute a predicted output vector comprising one or more outputs corresponding to one or more attributes of the plant or process, and wherein the predicted output vector is usable to manage the plant or process; and

wherein the non-linear portion comprises a function, wherein for each of the one or more inputs;

the function has at most one bend; and

as the input respectively approaches positive and negative infinity, the function asymptotically approaches respective lines of constant slope.

2. The system of claim 1, wherein the model is operable to be trained by a training process, wherein the training process is operable to:

a) receive training input data representative of inputs to the plant or process;

b) execute the model to generate predicted training output data for the plant or process corresponding to the training input data;

c) compare the predicted training output data to training output data representative of output of the plant or process to determine an error;

d) update the model in accordance with the determined error; and

e) perform a)-d) in an iterative manner to minimize the error.

3. The system of claim 2, wherein, to train the model, the training process is operable to train the model to optimize a specified objective function.

4. The system of claim 3, wherein the training process is operable to train the model to optimize the specified objective function subject to one or more constraints.

5. The system of claim 4, wherein the system further comprises an optimizer, wherein the one or more constraints comprise one or more of:

one or more hard constraints, wherein the one or more hard constraints comprise strict limitations on the training process in optimizing the objective function; and

one or more soft constraints, wherein the one or more soft constraints comprise a weighted penalty function comprised in the objective function.

6. The system of claim 5, wherein the optimizer comprises a non-linear programming optimizer.

7. The system of claim 4,

wherein the model comprises one or more gains, wherein each gain comprises a partial derivative of an output with respect to a respective input, wherein the one or more constraints comprise one or more gain constraints.

8. The system of claim 7, wherein each of the one or more inputs has a lower bound and an upper bound, wherein the gain constraints comprise respective constraints on the gain at the lower bound and/or the upper bound of the respective input.

9. The system of claim 4, wherein the one or more constraints comprise one or more soft constraints, and wherein the training process is operable to train the model via gradient descent.

10. The system of claim 1, wherein the model comprises a neural network.

11. The system of claim 1, wherein the input vector comprises a plurality of inputs, wherein the model includes no cross terms among the plurality of inputs.

12. The system of claim 1, wherein the input vector comprises a plurality of inputs, wherein the model comprises one or more cross terms among at least two of the plurality of inputs.

13. The system of claim 1, wherein the input vector comprises a plurality of inputs, wherein the function is of the form ln(1/(1+e^u)), wherein u is a function of the plurality of inputs and optionally specifies translation and/or scaling of the plurality of inputs.

14. The system of claim 1, wherein the system further comprises:

the plant or process; and

a module coupled to the plant or process;

wherein the module is operable to;

receive plant or process input data;

provide the plant or process input data to the model as input;

execute the model to generate predicted plant or process output based on the plant or process input data; and

wherein the predicted plant or process output is useable to manage the plant or process.

15. The system of claim 1, wherein the system further comprises:

the plant or process; and

an optimizer coupled to the plant or process;

wherein the optimizer is operable to utilize the model for optimization of the plant or process, wherein, to utilize the model for optimization of the plant or process, the optimizer is operable to;

receive desired plant or process output;

execute the model in reverse using the desired plant or process output as input to generate predicted plant or process inputs based on the desired plant or process output;

provide the predicted plant or process inputs to the plant or process as input to operate the plant or process to produce plant or process output substantially in accordance with the desired plant or process output.

16. The system of claim 15, wherein, to optimize the plant or process, the optimizer is operable to optimize a specified objective function.

17. The system of claim 16, wherein the optimizer is operable to optimize the specified objective function subject to one or more constraints.

18. The system of claim 17, wherein the one or more constraints comprise one or more of:

one or more hard constraints, wherein the one or more hard constraints comprise strict limitations on the training process in optimizing the objective function; and

one or more soft constraints, wherein the one or more soft constraints comprise a weighted penalty function comprised in the objective function.

19. The system of claim 18, wherein the optimizer comprises a non-linear programming optimizer.

20. The system of claim 17, wherein the one or more constraints comprise one or more soft constraints, and wherein the optimizer is operable to optimize the plant or process via gradient descent.

21. The system of claim 1, wherein the function comprises an integrated sigmoid function.

22. A computer-accessible memory medium that stores program instructions executable by a processor to perform:

a model receiving an input vector comprising one or more inputs, wherein the model comprises a representation of a plant or process, wherein the model comprises a linear portion and a non-linear portion, and wherein the non-linear portion comprises a function, and wherein for each of the one or more inputs;

the function has at most one bend; and

as the input respectively approaches positive and negative infinity, the function asymptotically approaches respective lines of constant slope;

computing predicted output corresponding to one or more attributes of the plant or process; and

storing the predicted output, wherein the predicted output is usable to manage the plant or process.

23. A method, comprising:

receiving one or more inputs to a model, wherein the model comprises a representation of a plant or process, wherein the model comprises a linear portion and a non-linear portion, and wherein the non-linear portion comprises a function, wherein the function comprises an integrated sigmoid function, and wherein for each of the one or more inputs;

the function has at most one bend; and

as the input respectively approaches positive and negative infinity, the function asymptotically approaches respective lines of constant slope;

computing predicted output corresponding to one or more attributes of the plant or process; and

storing the predicted output, wherein the predicted output is usable to manage the plant or process.

24. A system, comprising:

a model, comprising;

one or more inputs, operable to receive input data;

a linear portion;

a non-linear portion, wherein the non-linear portion comprises an integrated sigmoid function;

wherein the model comprises a representation of a plant or process; and

wherein the model is operable to receive an input vector comprising one or more inputs, and compute a predicted output vector comprising one or more outputs corresponding to one or more attributes of the plant or process, and wherein the predicted output vector is usable to manage the plant or process.