Performance of artificial neural network models in the presence of instrumental noise and measurement errors

• US 20030191728A1
• Filed: 03/27/2002
• Published: 10/09/2003
• Est. Priority Date: 03/27/2002
• Status: Active Grant

• Current Assignee: Council of Scientific & Industrial Research
• Original Assignee: Council of Scientific & Industrial Research
• Inventors: Dheshmukh, Sanjay Vasantrao, Lonari, Jayaram Budhaji, Kulkarni, Bhaskar Dattatray, Ravichandran, Sivaraman, Shenoy, Bhavanishankar, Valecha, Neelamkumar, Tambe, Sanjeev Shrikrishna
• Granted Patent: US 7,313,550 B2
• Time in Patent Office: Days
• US Class Current: 706/21
• CPC Class Codes: G06N 3/086 using evolutionary algorithms, e.g. genetic algorithms or genetic programming

Abstract

A method is described for improving the prediction accuracy and generalization performance of artificial neural network models in presence of input-output example data containing instrumental noise and/or measurement errors, the presence of noise and/or errors in the input-output example data used for training the network models create difficulties in learning accurately the nonlinear relationships existing between the inputs and the outputs, to effectively learn the noisy relationships, the methodology envisages creation of a large-sized noise-superimposed sample input-output dataset using computer simulations, here, a specific amount of Gaussian noise is added to each input/output variable in the example set and the enlarged sample data set created thereby is used as the training set for constructing the artificial neural network model, the amount of noise to be added is specific to an input/output variable and its optimal value is determined using a stochastic search and optimization technique, namely, genetic algorithms, the network trained on the noise-superimposed enlarged training set shows significant improvements in its prediction accuracy and generalization performance, the invented methodology is illustrated by its successful application to the example data comprising instrumental errors and/or measurement noise from an industrial polymerization reactor and a continuous stirred tank reactor (CSTR).

Claims

1. A method for improving the prediction accuracy and generalization performance of nonlinear artificial neural network models when the input-output data, known as the example set, available for constructing the network model, comprises instrumental noise and/or measurement errors, said method comprising the steps of:

(a) generating noise-superimposed enlarged input-output sample data set using computer simulations;

p1 (b) generating for each input-output pattern in the example set, M number of noise-superimposed sample input-output patterns (vectors) using computer simulations,

(c) generating noise-superimposed sample input-output patterns using noise tolerance values, which are specific to each input/output variable;

(d) generating Gaussian (normally) distributed random numbers using computer simulations to create noise-superimposed sample input-output patterns;

(e) determining the exact amount of Gaussian noise to be added to each input/output variable in the example set by using a stochastic search and optimization technique and

(f) using the computer generated noise-superimposed sample input-output patterns as the ‘

training set’

for constructing the nonlinear artificial neural network model;

2. A method according to claim 1, wherein the exact amount of Gaussian noise to be added to each input output variable of the example set as determined by the genetic algorithms, is globally (and not locally) optimal.

3. A method according to claim 1, wherein the example set is used as the ‘

test set’

for monitoring the generalization performance of the artificial neural network model.

4. A method according to claim 1, wherein the artificial neural network architecture is ‘

feed-forward’

, that is, the information flow within the network is unidirectional, from the input layer to the output layer.

5. A method according to claim 1, wherein the feed-forward neural network architecture includes the multilayer perceptron (MLP) networks, radial basis function networks (RBFN), and counterpropagation neural networks (CPNN).

6. A method according to claim 1, wherein the algorithms used for constructing or training the artificial neural network model includes error-back-propagation, conjugate gradient, Quickprop and RPROP.

7. A method according to claim 1, wherein the stochastic search and optimization technique used to optimize the noise tolerances refers to genetic algorithms and related methods, namely, simulated annealing (SA), simultaneous perturbation stochastic approximation (SPSA), evolutionary algorithms (EA) and memetic algorithms (MA).

8. A method according to claim 1, wherein an enlarged noise-superimposed sample input-output data set is created using computer simulations from the small-sized example input-output set.