Neural-network based surrogate model construction methods and applications thereof

• US 8,065,244 B2
• Filed: 03/13/2008
• Issued: 11/22/2011
• Est. Priority Date: 03/14/2007
• Status: Active Grant

• Current Assignee: Halliburton Energy Services Incorporated (Halliburton Company)
• Original Assignee: Halliburton Energy Services Incorporated (Halliburton Company)
• Inventors: Stephenson, Stanley, Zhong, Allan, Hamid, Syed, Chen, Dingding
• Primary Examiner(s): FERNANDEZ RIVAS, OMAR F
• Application Number:
  US12/048,045
• Publication Number:
  US 20080228680A1
• Time in Patent Office: 1,349 Days
• Field of Search: None
• US Class Current: 706/15
• CPC Class Codes: B33Y 80/00 Products made by additive manufacturing
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

Various neural-network based surrogate model construction methods are disclosed herein, along with various applications of such models. Designed for use when only a sparse amount of data is available (a “sparse data condition”), some embodiments of the disclosed systems and methods: create a pool of neural networks trained on a first portion of a sparse data set; generate for each of various multi-objective functions a set of neural network ensembles that minimize the multi-objective function; select a local ensemble from each set of ensembles based on data not included in said first portion of said sparse data set; and combine a subset of the local ensembles to form a global ensemble. This approach enables usage of larger candidate pools, multi-stage validation, and a comprehensive performance measure that provides more robust predictions in the voids of parameter space.

Claims

1. A modeling system that comprises:

a memory; and

a processor coupled to the memory and configured to execute software stored in said memory, wherein said software configures the processor to;

create a pool of neural networks trained on a portion of a data set;

for each of various coefficient settings for a multi-objective function;

apply selective evolution subject to the multi-objective function with that coefficient setting to obtain a corresponding group of neural network ensembles; and

select a local ensemble from each said group of neural network ensembles, wherein the selection is based on data not included in said portion of the data set;

combine a plurality of the local ensembles to form a global ensemble of local ensembles; and

provide a perceptible output based at least in part on a prediction by the global ensemble.

2. The system of claim 1, wherein as part of creating the pool of neural networks, the software configures the processor to:

form multiple training data sets from said portion of the data set; and

train neural networks having different architectures with the multiple training data sets.

3. The system of claim 1, wherein the multi-objective function is based at least in part on a measure of the ensemble'"'"'s mean square error (EMSE) over said portion of the data set.

4. The system of claim 3, wherein the multi-objective function is further based at least in part on a measure of network ambiguity averaged over said portion of the data set.

5. The system of claim 1, wherein as part of forming a global ensemble, the software configures the processor to evaluate global ensemble candidates based at least in part on a measure of prediction error over the entire data set.

6. The system of claim 1, wherein as part of forming a global ensemble, the software configures the processor to evaluate global ensemble candidates based at least in part on a measure of network ambiguity over a data subspace including values not in the data set.

7. The system of claim 1, wherein as part of forming a global ensemble, the software configures the processor to evaluate global ensemble candidates based at least in part on a measure of smoothness or consistency over a data subspace.

8. The system of claim 1, wherein as part of forming a global ensemble, the software configures the processor to graphically render prediction subspaces of global ensemble candidates for evaluation of those candidates.

9. The system of claim 1, wherein as part of providing a perceptible output, the software configures the processor to use the global ensemble to identify at least one potential solution for submission to a subsequent modeling process.

10. The system of claim 1, wherein as part of providing a perceptible output, the software configures the processor to use the global ensemble to find within a parameter space a solution that is at least locally optimum.

11. The system of claim 1, wherein the perceptible output is at least one specification value for a product.

12. The system of claim 1, wherein the perceptible output is a control signal for a regulated process.

13. A computer-based modeling process that comprises:

obtaining a data set having output values associated with input values;

partitioning the data set into primary and secondary subsets;

training a pool of neural networks without using data from the secondary subset;

developing a group of neural network ensembles using different objective functions;

selecting local ensembles from the group using data from the secondary subset;

forming a global ensemble having multiple local ensembles; and

providing a perceptible output based at least in part on a prediction by the global ensemble.

14. The process of claim 13, wherein said combining multiple local ensembles includes evaluating global ensemble candidates based at least in part on a measure of network ambiguity over a data subspace including values not in the data set.

15. The process of claim 13, wherein said combining multiple local ensembles includes evaluating global ensemble candidates based at least in part on a measure of prediction smoothness over a parameter space.

16. The process of claim 13, wherein said providing a perceptible output includes using the global ensemble to find within a parameter space a solution that is at least locally optimum.

17. A method that comprises:

determining a system'"'"'s response to a limited set of input parameter values;

deriving a system model that predicts the system'"'"'s response over a larger set of input parameter values, wherein the system model includes a neural network ensemble comprising multiple local neural network ensembles, each local neural network ensemble selected from a corresponding set of neural network ensembles developed based on a particular weighting for a multi-objective function, wherein said multi-objective function is evaluated based on a first portion of the limited set of input parameter values, and wherein said selection is made based on input parameter values held out from said first portion; and

storing or displaying a system response predicted by the system model.

18. The method of claim 17, wherein said neural network ensemble is selected from a group of global ensemble candidates that are evaluated based at least in part on a measure of network ambiguity over a data subspace.

19. The method of claim 17, wherein said neural network ensemble is selected from a group of global ensemble candidates that are evaluated based at least in part on a measure of smoothness over a data subspace.

20. The method of claim 17, further comprising producing a product having a characteristic based at least in part on the predicted system response.