Markov decision process-based support tool for reservoir development planning

• US 8,775,347 B2
• Filed: 01/30/2009
• Issued: 07/08/2014
• Est. Priority Date: 04/18/2008
• Status: Active Grant

• Current Assignee: Exxonmobil Upstream Research Company (Exxon Mobil Corporation)
• Original Assignee: Exxonmobil Upstream Research Company (Exxon Mobil Corporation)
• Inventors: Goel, Vikas, Furman, Kevin C.
• Primary Examiner(s): Chang, Li-Wu
• Assistant Examiner(s): Kim, David H
• Application Number:
  US12/864,971
• Publication Number:
  US 20100325075A1
• Time in Patent Office: 1,985 Days
• Field of Search: 703/10
• US Class Current: 706/46
• CPC Class Codes: G01V 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A Markov decision process-based support tool for reservoir development planning can comprise a source of input data, an optimization model, a high fidelity model for simulating the reservoir, and one or more solution routines interfacing with the optimization model. The optimization model can consider unknown parameters having uncertainties directly within the optimization model. The model incorporates the flexibility that a decision-maker has in the real world and allows the decision-maker to adjust the decisions based on new information. The model can systematically address uncertain data, for example comprehensively or even taking all uncertain data into account. Accordingly, the optimization model can provide flexible or robust solutions that remain feasible over an uncertainty space. Once the reservoir model is optimized, final development plans may be generated.

Claims

1. A method for reservoir development planning, comprising the steps of:

receiving data relevant to reservoir development;

processing the received data by using a computer-implemented Markov decision process-based model,

wherein the using the computer-implemented Markov decision process-based model includes

defining a plurality of stages, each stage representing a discrete step in time;

defining a plurality of states in each stage, each state representing a potential state of the reservoir development plan, and

determining a future state by using a probability that the reservoir development transitions from a current state in a current stage to the future state in a subsequent stage, the probability being determined by the current state of the reservoir development and an action to be taken in the current stage that causes the transition to the subsequent stage; and

generating a reservoir development plan in response to processing the received data via the computer-implemented Markov decision process-based model.

2. The method of claim 1, wherein the generating step comprises generating the reservoir development plan according to an uncertainty space associated with the received data.

3. The method of claim 1, wherein the computer-implemented Markov decision process-based model comprises optimizing at least some aspect of the reservoir development plan based on the received data and an uncertainty space.

4. The method of claim 1, wherein the step of receiving data comprises receiving known data parameters and uncertain data parameters, and

wherein processing the received data via the computer-implemented Markov decision process-based model comprises processing the data with a Markov decision process-based model that incorporates the uncertain data parameters.

5. The method of claim 1, wherein an uncertainty space is associated with the received data, and

wherein processing the received data via the computer-implemented Markov decision process-based model comprises considering the uncertainty space in entirety.

6. The method of claim 1, wherein a decision-maker is allowed to undertake one or more corrective decisions at each of the plurality of stages within the Markov decision process-based model.

24. The method of claim 1, wherein the probability includes a plurality of transition probabilities for transitioning from the current state in the current stage to possible future states in the subsequent stage.

7. A method for reservoir development planning, comprising the steps of:

receiving data relevant to reservoir development planning, wherein uncertainty is associated with the data;

processing the received data by using a computer-implemented Markov decision process-based model,

wherein the using the computer-implemented Markov decision process-based model includes

defining a plurality of stages, each stage representing a discrete step in time;

defining a plurality of states in each stage, each state representing a potential state of the reservoir development plan, and

determining a future state by using a probability that the reservoir development transitions from a current state in a current stage to the future state in a subsequent stage, the probability being determined by the current state of the reservoir development and an action to be taken in the current stage that causes the transition to the subsequent stage;

producing at least some portion of a reservoir development plan in response to processing the received data with the computer-based optimization model that incorporates the uncertainty; and

undertaking one or more corrective decisions as the uncertainty unfolds over time.

8. The method of claim 7, wherein incorporating the uncertainty in the Markov decision process-based model comprises capturing tradeoffs across a plurality of realizations of the uncertainty.

9. The method of claim 7, wherein the step of producing at least some portion of the reservoir development plan in response to processing the received data with the computer-based optimization model that incorporates the uncertainty comprises

achieving feasibility of an optimization problem across an entire uncertainty space.

10. The method of claim 7, wherein the step of producing at least some portion of the reservoir development plan in response to processing the received data with the computer-based optimization model that incorporates the uncertainty comprises

systemically processing uncertain data within the Markov decision process-based model.

11. A method for decision support regarding development of petroleum resources, comprising the steps of:

receiving a plurality of data elements regarding a decision about developing a petroleum reservoir, wherein a respective characterization of uncertainty is associated with each of the data elements; and

outputting a recommendation for the decision in response to processing each of the data elements and each of the respective characterizations of uncertainty by using a computer-implemented Markov decision process-based model,

wherein the using the computer-implemented Markov decision process-based model includes;

defining a plurality of stages, each stage representing a discrete step in time;

defining a plurality of states in each stage, each state representing a potential state of the reservoir development plan, and

determining a future state by using a probability that the reservoir development transitions from a current state in a current stage to the future state in a subsequent stage, the probability being determined by the current state of the reservoir development and an action to be taken in the current stage that causes the transition to the subsequent stage.

12. The method of claim 11, wherein the Markov decision process-based model incorporates each of the respective characterizations of uncertainty.

13. The method of claim 11, wherein processing each of the data elements and each of the respective characterizations of uncertainty with the computer-based Markov decision process-based model comprises covering an uncertainty space.

14. A computer-based method of optimizing development planning for a reservoir, comprising:

providing input data that comprises uncertainty;

generating a first simulation of the reservoir using a high fidelity model for reservoir or surface facility behavior utilizing the input data, wherein the first simulation generates first high fidelity output data;

generating a low fidelity model for reservoir or surface facility behavior using the input data and the first high fidelity output data, wherein the low fidelity model generates a prediction;

optimizing a reservoir development planning model utilizing the input data and the low fidelity model for reservoir or surface facility behavior, wherein the reservoir development planning model generates reservoir development planning output data, wherein the optimizing the reservoir development planning model comprises using a computer-implemented Markov decision process-based model, and the using the computer-implemented Markov decision process-based model includes;

defining a plurality of stages, each stage representing a discrete step in time;

defining a plurality of states in each stage, each state representing a potential state of the reservoir development plan, and

determining a future state by using a probability that the reservoir development transitions from a current state in a current stage to the future state in a subsequent stage, the probability being determined by the current state of the reservoir development and an action to be taken in the current stage that causes the transition to the subsequent stage;

performing a second simulation of the reservoir using the high fidelity model for reservoir or surface facility behavior utilizing the input data and the reservoir development planning output data, wherein the second simulation generates second high fidelity output data;

comparing the second high fidelity output data with the prediction of the low fidelity model;

repeating the steps comprising generating a low fidelity model for reservoir or surface facility behavior, optimizing a reservoir development planning model, performing a second simulation of the reservoir using the high fidelity model for reservoir or surface facility behavior, and comparing the second high fidelity output data with the prediction of the low fidelity model until the second high fidelity output data is substantially consistent with the prediction of the low fidelity model.

15. The method of claim 14, further comprising:

generating a development plan in response to the second high fidelity output data being substantially consistent with the prediction of the low fidelity model.

16. The method of claim 14, wherein a solution routine, interfacing with the Markov decision process-based model, assists the Markov decision process-based model to optimize development planning for the reservoir.

17. The method of claim 14, wherein the Markov decision process-based model comprises a solution routine for assisting with optimizing development planning for the reservoir.

18. A method of producing hydrocarbons from a subterranean reservoir, comprising:

generating a reservoir development planning system based on input data;

optimizing the reservoir development planning system according to an uncertainty space, wherein the reservoir development planning system is optimized using a computer-implemented Markov decision process-based model,

wherein the using the computer-implemented Markov decision process-based model includes;

defining a plurality of stages, each stage representing a discrete step in time;

defining a plurality of states in each stage, each state representing a potential state of the reservoir development plan, and

determining a future state by using a probability that the reservoir development transitions from a current state in a current stage to the future state in a subsequent stage, the probability being determined by the current state of the reservoir development and an action to be taken in the current stage that causes the transition to the subsequent stage; and

producing hydrocarbons from the reservoir according to output from the optimized reservoir development planning system.

19. The method of claim 18, wherein the input data comprises deterministic components and nondeterministic components, and

wherein the step of optimizing the reservoir development planning system according to the uncertainty space comprises

considering each of the nondeterministic components with the Markov decision process-based model.

20. The method of claim 18, wherein the Markov decision process-based model incorporates uncertainty of the input data.

21. The method of claim 18, wherein the uncertainty space specifies inherent uncertainty of the input data.

22. The method of claim 18, wherein the reservoir development planning system comprises:

a Markov decision process-based model comprising a low fidelity reservoir model; and

a high fidelity reservoir model,

wherein the high fidelity model accepts one or more parameter input data from the Markov decision process-based model and provides one or more reservoir or surface facility property input data to the Markov decision process-based model.

23. The method of claim 18, wherein optimizing the reservoir development planning system comprises:

simulating operation of the reservoir using a high fidelity model; and

simulating operation of the reservoir using a low fidelity model.