Hybrid neural network and support vector machine method for optimization

• US 6,961,719 B1
• Filed: 01/07/2002
• Issued: 11/01/2005
• Est. Priority Date: 01/07/2002
• Status: Expired due to Fees

• Current Assignee: USA As Represented By The Administrator Of The NASA
• Original Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Inventors: Rai, Man Mohan
• Primary Examiner(s): Hirl, Joseph P.
• Application Number:
  US10/043,044
• Time in Patent Office: 1,394 Days
• Field of Search: 706/21, 706/19, 706/25
• US Class Current: 706/21
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

System and method for optimization of a design associated with a response function, using a hybrid neural net and support vector machine (NN/SVM) analysis to minimize or maximize an objective function, optionally subject to one or more constraints. As a first example, the NN/SVM analysis is applied iteratively to design of an aerodynamic component, such as an airfoil shape, where the objective function measures deviation from a target pressure distribution on the perimeter of the aerodynamic component. As a second example, the NN/SVM analysis is applied to data classification of a sequence of data points in a multidimensional space. The NN/SVM analysis is also applied to data regression.

Claims

1. A computer implemented machine learning method for use in engineering applications, including but not limited to optimizing designs, classifying data and generating regression estimates, that is a hybrid of neural net (“

NN”

) analysis and support vector machine (“

SVM”

) analysis, the method comprising;

(a) providing an NN component, having an input layer and a hidden layer and an input vector space, where the NN component automatically generates coordinates in a feature vector space, and providing an SVM component that utilizes the feature vector space;

(b) selecting a group of parameters and combinations of parameters and providing a feature space coordinate, in the feature vector space, for each selected parameter and selected parameter combination in the input space for use in at least one of optimizing a design, controlling a physical or chemical process, classifying data and generating regression estimates for a collection of the data;

(c) providing at least one vector of candidate parameter values for each of the group of parameters in the input space;

(d) providing initial values for connection weights between the input layer and the hidden layer for the NN component;

(e) computing hidden layer output signals, corresponding to the connection weight values, for each of the parameter value vectors;

(f) using at least one hidden layer output signal as a feature space coordinate for the SVM component;

(g) determining inner product values of a selected number of at least two feature space coordinates;

(h) providing a Lagrange functional using the determined inner product values;

(i) providing at least two constraints, expressed in terms of Lagrange multipliers and input vector space data;

(j) minimizing the Lagrange functional, subject to at least one selected constraint, to obtain Lagrange multiplier values corresponding to the minimized Lagrange functional;

(k) computing a training error, using the connection weights for the NN component and the Lagrange multiplier values for the SVM component;

(l) when the computed training error is greater than a selected threshold value, changing at least one of the connection weights and repeating steps (e)–

(k) at least once, wherein at least one feature space coordinate value changes automatically in response to change in the at least one connection weight; and

(m) when the computed training error is not greater than the threshold value, interpreting the NN component with the associated connection weights and the SVM component with the associated Lagrange multipliers as a trained NN/SVM system.

2. The method of claim 1, further comprising augmenting said inner product value with at least one user-specified inner product value in said SVM component.

3. The method of claim 1, further comprising:

providing a collection of N data points in an M-dimensional space for said input space, where M≧

2 and N≧

2, and where each data point is assigned an indicium associated with one of at least first and second mutually exclusive sets; and

applying the method of claim 1 for determination of a separation surface in the M-dimensional space that separates the data points into at least first and second mutually exclusive regions that contain substantially all data points in the first set and in the second set, respectively.

4. The method of claim 3, further comprising providing a visually perceptible view of at least a portion of said separation surface in at least two dimensions.

5. The method of claim 1, wherein said use in engineering applications comprises design of a airfoil representing a wing or other control surface of an aircraft.

6. The method of claim 5, further comprising:

providing an optimization method; and

using the optimization method in at least one of steps (e) through (k) to obtain at least one of said connection weight values, said Lagrange multiplier values and said inner product value for said aircraft airfoil.

7. The method of claim 5, further comprising determining an optimized design for said aircraft airfoil by applying a response surface analysis to said design, using said trained NN/SVM system.

8. The method of claim 7, further comprising providing a selected optimization procedure in determining said optimized design.

9. The method of claim 5, further comprising computing said training error by a process comprising:

providing a collection of at least two desired pressure values, with each desired pressure value corresponding to a location on said airfoil;

computing a pressure value at each of the selected locations on said airfoil, using said minimized Lagrangian functional; and

computing said training error as a sum of magnitudes of differences between the desired pressure value and the computed pressure value at each of the selected locations on said airfoil.

10. The method of claim 5, further comprising computing said airfoil by a process comprising:

(n) providing a collection of a least two desired pressure values, with each desired pressure value corresponding to a location on said airfoil;

(o) providing an initial airfoil shape;

(p) computing a pressure distribution for said airfoil for at least one perturbation in the initial airfoil shape;

(q) representing variation of pressure with the at least one perturbation in the initial airfoil shape, using said NN component and said SVM component, wherein said NN input vector corresponds to the at least one perturbation in the initial airfoil shape and an SVM output signal corresponds to at least one change in the pressure distribution;

(r) computing an objective function value, which is to be minimized for said airfoil design optimization, as a sum of magnitudes of a power of differences between the desired pressure value and a pressure value using said NN component and said SVM component;

(s) when the objective function value is greater than a selected threshold value, repeating steps (p)–

(r) at least once; and

(t) when the objective function not greater than the threshold value, interpreting this condition as indicating that an optimal airfoil design is identified.

11. The method of claim 1, wherein said use in engineering applications comprises design of a airfoil representing at least one turbine or compressor airfoil.