Method and apparatus for computer-supported generation of at least one artificial training data vector for a neural network

• US 6,282,529 B1
• Filed: 03/03/1999
• Issued: 08/28/2001
• Est. Priority Date: 09/03/1996
• Status: Expired due to Fees

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Zimmermann, Hans-Georg, Neuneier, Ralf
• Primary Examiner(s): Powell, Mark R.
• Assistant Examiner(s): STARKS, WILBERT L
• Application Number:
  US09/254,298
• Time in Patent Office: 909 Days
• Field of Search: 706/23, 706/15, 706/16, 382/157
• US Class Current: 706/15
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

A method and apparatus for computer-supported generation of at least one artificial training data vector for a neural network is provided wherein a residual error is determined after a training of a neural network has occurred. A backward error is then determined from the residual error. Artificial training data vectors are generated from a statistical random process that is based on a statistical distribution, such that the respective backward error for an input of the neural network is taken into consideration.

Claims

1. A method for computer supported generation of at least one artificial training data vector for a neural network, comprising the steps of:

training the neural network with training data vectors of a training data set;

determining a residual error after the step of training the neural network;

determining a backward error from the residual error for at least one input of the neural network;

generating a statistical distribution allocated to the at least one input, the statistical distribution being dependent on a size of the respective backward error; and

generating the at least one artificial training data vector given the respective allocation of the statistical distribution to the at least one input, wherein the at least one artificial training data vector exhibits the same properties as an associated training data vector of the training data set.

2. A method as claimed in claim 1, wherein the backward error is determined with a gradient descent method.

3. A method as claimed in claim 2, wherein the gradient descent method is performed via a back-propagation method.

4. A method as claimed in claim 1, wherein the statistical distribution is generated via a uniform distribution, a distribution width of the uniform distribution being dependent on the backward error.

5. A method as claimed in claim 4, wherein, given an online approximation of the neural network, the statistical distribution is adapted to a newly determined backward error according to an arbitrary plurality of new training data vectors.

6. A method as claimed in claim 1, wherein the statistical distribution is generated via a Gaussian distribution, a distribution width of the Gaussian distribution being dependent on the backward error.

7. A method as claimed in claim 6, wherein, given an online approximation of the neural network, the statistical distribution is adapted to a newly determined backward error according to an arbitrary plurality of new training data vectors.

8. A method as claimed in claim 1, wherein the method is utilized for modeling a nonlinear function that is described with a low number of training data vectors.

9. A method as claimed in claim 1, wherein the method is utilized in the analysis of a financial market.

10. An apparatus for the computer supported generation of at least one artificial training data vector for a neural network, the apparatus including a calculating unit which comprises:

means for training the neural network with training data vectors of a training data set;

means for determining a residual error after training the neural network;

means for determining a backward error from the residual error for at least one input of the neural network;

means for generating a statistical distribution allocated to the at least one input, the statistical distribution being dependent on a size of the respective backward error; and

means for generating the at least one artificial training data vector given the respective allocation of the statistical distribution to the at least one input, wherein the at least one artificial training data vector exhibits the same properties as an associated training data vector of the training data set.

11. An apparatus as claimed in claim 10, wherein the backward error is determined with a gradient descent method.

12. An apparatus as claimed in claim 11, wherein the gradient descent method is performed via a back-propagation method.

13. An apparatus as claimed in claim 10, wherein the statistical distribution is generated via a uniform distribution, a distribution width of the uniform distribution being dependent on the backward error.

14. An apparatus as claimed in claim 13, wherein, given an online approximation of the neural network, the statistical distribution is adapted to a newly determined backward error according to an arbitrary plurality of new training data vectors.

15. An apparatus as claimed in claim 10, wherein the statistical distribution is generated via a Gaussian distribution, a distribution width of the Gaussian distribution being dependent on the backward error.

16. An apparatus as claimed in claim 15, wherein, given an online approximation of the neural network, the statistical distribution is adapted to a newly determined backward error according to an arbitrary plurality of new training data vectors.

17. An apparatus as claimed in claim 10, wherein the apparatus is utilized for modeling a non-linear function that is described with a low number of training data vectors.

18. An apparatus as claimed in claim 10, wherein the apparatus is utilized in the analysis of a financial market.