Ensembles of neural networks with different input sets

• US 7,613,665 B2
• Filed: 06/24/2005
• Issued: 11/03/2009
• Est. Priority Date: 06/24/2005
• Status: Active Grant

• Current Assignee: Halliburton Energy Services Incorporated (Halliburton Company)
• Original Assignee: Halliburton Energy Services Incorporated (Halliburton Company)
• Inventors: Hamid, Syed, Quirein, John A., Chen, Dingding, Smith, Harry D. Jr., Grable, Jeffery L.
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): Kennedy; Adrian L
• Application Number:
  US11/165,892
• Publication Number:
  US 20070011114A1
• Time in Patent Office: 1,593 Days
• Field of Search: 706/16, 706/15, 706/20, 700/47, 700/48
• US Class Current: 706/16
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/086 using evolutionary algorithms, e.g. genetic algorithms or genetic programming

Abstract

Methods of creating and using robust neural network ensembles are disclosed. Some embodiments take the form of computer-based methods that comprise receiving a set of available inputs; receiving training data; training at least one neural network for each of at least two different subsets of the set of available inputs; and providing at least two trained neural networks having different subsets of the available inputs as components of a neural network ensemble configured to transform the available inputs into at least one output. The neural network ensemble may be applied as a log synthesis method that comprises: receiving a set of downhole logs; applying a first subset of downhole logs to a first neural network to obtain an estimated log; applying a second, different subset of the downhole logs to a second neural network to obtain an estimated log; and combining the estimated logs to obtain a synthetic log.

Claims

1. A method of synthesizing a well log, the method comprising:

receiving a set of input signals that represent measurements of downhole formation characteristics;

applying a first subset of the set of input signals to a first neural network to obtain one or more estimated logs;

applying a second subset of the set of input signals to a second neural network to obtain one or more estimated logs, wherein the first and second subsets are distinct in that one subset has at least one input signal not included by the other; and

combining corresponding ones of the one or more estimated logs from the first and second neural networks to output one or more synthetic logs to display to a user.

2. The method of claim 1, wherein the set of input signals represents a set of cased hole logs derived from measurements by a pulsed neutron tool.

3. The method of claim 2, wherein the pulsed neutron tool is a pulsed neutron capture tool.

4. The method of claim 2, wherein the set of input signals is derived from measurements by at least one of the following:

full wave sonic logging tools, natural gamma ray spectroscopy tools, cased hole resistivity logging tools, cased hole neutron logging tools, and production logging tools.

5. The method of claim 1, wherein the set of input signals is derived from measurements by at least one of the following:

full wave sonic logging tools, natural gamma ray spectroscopy tools, cased hole resistivity logging tools, cased hole neutron logging tools, and production logging tools.

6. The method of claim 1, wherein the one or more synthetic logs comprise at least one of a formation density log, a formation resistivity log, a neutron porosity log, and a magnetic resonance imaging log.

7. The method of claim 1, wherein the first and second neural networks have different values of an architectural parameter.

8. The method of claim 1, wherein the first and second neural networks are backpropagation networks with different numbers of hidden nodes.

9. The method of claim 1, wherein the second subset is a proper subset of the first subset.

10. The method of claim 1, wherein the first subset equals the set of input signals.

11. A computer-based method that generates a trained neural network ensemble, the method comprising:

receiving a set of available inputs;

receiving training data comprising values for the available inputs and corresponding values for at least one output;

training at least one neural network for each of at least two different subsets of the set of available inputs, wherein the at least two different subsets are distinct in that one subset has at least one input signal from a sensor type not included in the other subset;

providing at least two trained neural networks having different subsets of the available inputs as components of a neural network ensemble that transforms said available inputs into said at least one output, wherein said components are selected at least in part based on a measure of negative correlation between the components; and

plotting said at least one output from the neural network ensemble as a function of at least one of time, depth, and position.

12. The computer-based method of claim 11, wherein the set of available inputs is a set of cased hole logs that can be derived from measurements by a pulsed neutron tool.

13. The computer-based method of claim 12, wherein the set of available input signals further includes cased hole logs that can be derived from measurements by at least one of the following:

a full wave sonic logging tool, a natural gamma ray spectroscopy tool, a cased hole resistivity logging tool, a cased hole neutron logging tool, and a production logging tool.

14. The computer-based method of claim 11, wherein at least one output comprises at least one of a formation density log, a formation resistivity log, a neutron porosity log, and a magnetic resonance imaging log.

15. The computer-based method of claim 11, wherein said training results in a pool of trained neural networks, and wherein said providing includes:

forming an initial population of neural network ensembles, said ensembles comprising neural networks from the pool;

evolving the population of neural network ensembles to improve a fitness measure; and

selecting an ensemble from the evolved population.

16. The computer-based method of claim 15, wherein said evolving is based on a genetic algorithm.

17. The computer-based method of claim 15, wherein the fitness measure is determined from at least one of the following:

a combination of a performance measure, a complexity measure, and a negative correlation measure.

18. The computer-based method of claim 11, wherein said training results in a pool of trained neural networks, said pool having diversity in neural network inputs and neural network architectures.

19. The computer-based method of claim 18, wherein the neural networks in the pool are backpropagation networks, and wherein the architectural diversity resides in a hidden layer size.

20. The computer-based method of claim 11, wherein the training includes training at least one neural network for each of a series of different input subsets, the input subsets each being a proper subset of all subsequent input subsets in the series.

21. The computer-based method of claim 20, wherein the number of input subsets in the series is at least equal to the number of available inputs.

22. The computer-based method of claim 11, wherein the training includes determining for each input subset size a best subset of available inputs.

23. The computer-based method of claim 22, wherein the training includes constraining the input subsets tested for a best subset to those subsets that include the best subsets of smaller size.

24. The computer-based method of claim 22, wherein the training includes constraining the input subsets tested for a best subset to those subsets that are proper subsets of the best subsets of larger size.

25. The computer-based method of claim 11, wherein said training results in a locally ranked set of trained candidate neural networks.

26. The computer-based method of claim 11, wherein said training results in a globally ranked set of trained candidate neural networks.

27. The computer-based method of claim 11, wherein said training results in a set of trained neural networks, and wherein the method further comprises setting a candidate neural network pool size based on a number of trained neural networks in the set satisfying a performance measure.

28. The computer-based method of claim 27, wherein the performance measure is a prediction error threshold determined from a minimum prediction error of a set of trained neural networks for a best input subset.

29. An information storage medium that, when placed in operable relation to a computer, provides the computer with software that generates a trained neural network ensemble, the software comprising:

a training process that generates a set of neural networks having diversity in inputs and in complexity, wherein diversity in inputs requires that at least one of the neural networks in the set operates on a different combination of input signals than another of the neural networks in the set;

a selection process that identifies a combination of neural networks from the set having a desirable fitness measure, said fitness measure being based at least in part on a measure of negative correlation for each neural network in the combination;

a transform process that applies the combination of neural networks in ensemble fashion to a set of inputs to synthesize at least one output; and

a process to display said at least one output.

30. The medium of claim 29, wherein said at least one output comprises at least one of a formation density log, a formation resistivity log, a neutron porosity log, and a magnetic resonance imaging log.

31. The medium of claim 29, wherein the set of neural networks is locally ranked.

32. The medium of claim 29, wherein the set of neural networks is globally ranked.

33. The medium of claim 29, wherein the selection process forms a pool of candidate neural networks from best-performing neural networks in the set of neural networks after determining a pool size, and wherein the selection process determines a pool size based on a number of neural networks in the set that satisfy a performance measure.

34. The medium of claim 33, wherein the performance measure is a prediction error threshold determined from a minimum prediction error by a set of neural networks trained from a best input subset.