Drilling automation using stochastic optimal control

• US 9,995,129 B2
• Filed: 10/21/2013
• Issued: 06/12/2018
• Est. Priority Date: 10/21/2013
• Status: Active Grant

• Current Assignee: Halliburton Energy Services Incorporated (Halliburton Company)
• Original Assignee: Halliburton Energy Services Incorporated (Halliburton Company)
• Inventors: Dykstra, Jason, Xue, Yuzhen
• Primary Examiner(s): Cao, Chun
• Application Number:
  US15/023,105
• Publication Number:
  US 20160230530A1
• Time in Patent Office: 1,695 Days
• Field of Search: 700275
• CPC Class Codes: E21B 44/00 Automatic control systems specially adapted for drilling operations, i.e. self-operating systems which function to carry out or modify a drilling operation without intervention of a human operator, e.g. computer-controlled drilling systems; Systems specially adapted for monitoring a plurality of drilling variables or conditions
• CPC Class Codes: E21B 44/04 in response to the torque of the drive ; Measuring drilling torque E21B44/06 takes precedence; measuring stresses in a well bore pipe E21B47/007
• CPC Class Codes: E21B 44/06 in response to the flow or pressure of the motive fluid of the drive
• CPC Class Codes: E21B 45/00 Measuring the drilling time or rate of penetration
• CPC Class Codes: G05B 13/041 in which a variable is automatically adjusted to optimise the performance
• CPC Class Codes: G06F 17/11 for solving equations , e.g. nonlinear equations, general mathematical optimization problems optimization specially adapted for a specific administrative, business or logistic context G06Q10/04

Abstract

An example method for drilling automation may comprise generating a model of a drilling system based, at least in part, on a first set of downhole measurements. The model may accept drilling parameters of the drilling system as inputs. A rate of penetration for the drilling system may be determined based, at least in part on the model. The model may be simulated using a first set of values for the drilling parameters, and a control policy for the drilling system may be calculated based, at least in part, on the rate of penetration and the results of the simulation. A control signal to the drilling system may be generated based, at least in part, on the control policy.

Claims

1. A method for drilling automation, comprising:

generating a model of a drilling system based, at least in part, on a first set of downhole measurements, the model accepting drilling parameters of the drilling system as inputs;

determining a rate of penetration for the drilling system based, at least in part, on the model, wherein determining the rate of penetration for the drilling system comprises determining a maximum rate of penetration for the drilling system;

simulating the model using a first set of values for the drilling parameters, wherein simulating the model using the first set of values for the drilling parameters comprises generating a second set of values for the WOB, rotation rate, and flow rate that correspond to the first set of values;

calculating a control policy for the drilling system based, at least in part, on the rate of penetration and the results of the simulation, wherein calculating the control policy for the drilling system comprises comparing the second set of values to the values of a WOB, a rotation rate, and a flow rate that correspond to the maximum rate of penetration; and

generating a control signal to the drilling system based, at least in part, on the control policy to alter one or more drilling operations of the drilling system.

2. The method of claim 1, wherein generating the model of the drilling system comprises generating a space state model of the drilling system.

3. The method of claim 1, wherein

the drilling parameters of the drilling system comprise

a hook load of a hook of the drilling system;

a pump rate of a pump of the drilling system; and

a torque value of a top drive of the drilling system; and

the model generates as an output at least one of

a weight on a drill bit (WOB) of the drilling system;

a rotation rate of the drill bit; and

a flow rate of drilling fluid through the drilling system.

4. The method of claim 3, wherein generating the control signal to the drilling system comprises generating a control signal corresponding to at least one of the drilling parameters.

5. The method of claim 3, wherein determining the maximum rate of penetration for the drilling system comprises determining values of the WOB, rotation rate, and flow rate that correspond to the maximum rate of penetration.

6. The method of claim 1, wherein calculating the control policy for the drilling system further comprises

tracking the differences between the second set of values and the values of the WOB, rotation rate, and flow rate that correspond to the maximum rate of penetration using a cost function;

calculating a value function corresponding to the lowest average output of the cost function;

calculating a control input for each of the state of the drilling system using the value function; and

generating a look-up table containing the control inputs and the states of the drilling system.

7. The method of claim 6, wherein generating the control signal to the drilling system based, at least in part, on the control policy comprises

generating a real-time estimation of a state of the drilling system;

selecting a control input from the look-up table that corresponds to the estimated state;

generating the control signal for the drilling system using the control input.

8. The method of claim 1, further comprising

receiving a second set of downhole measurements;

generating a second model of the drilling system based, at least in part, on the second set of downhole measurements;

calculating a second control policy based, at least in part, on the second model; and

generating a second control signal to the drilling system based, at least in part, on the second control policy.

9. An apparatus for drilling automation, comprising:

a processor;

a memory device coupled to the processor, wherein the memory device contains a set of instructions that, when executed by the processor, cause the processor to

generate a model of a drilling system based, at least in part, on a first set of downhole measurements, the model accepting drilling parameters of the drilling system as inputs;

determine a rate of penetration for the drilling system based, at least in part, on the model, wherein determining the rate of penetration for the drilling system comprises determining a maximum rate of penetration for the drilling system;

simulate the model using a first set of values for the drilling parameters, wherein simulating the model using the first set of values for the drilling parameters comprises generating a second set of values for the WOB, rotation rate, and flow rate that correspond to the first set of values;

calculate a control policy for the drilling system based, at least in part, on the rate of penetration and the results of the simulation, wherein calculating the control policy for the drilling system comprises comparing the second set of values to the values of a WOB, a rotation rate, and a flow rate that correspond to the maximum rate of penetration; and

generate a control signal to the drilling system based, at least in part, on the control policy to alter a drilling operation of the drilling system.

10. The apparatus of claim 9, wherein the set of instructions that cause the processor to generate the model of the drilling system further causes the processor to generate a space state model of the drilling system.

11. The apparatus of claim 9, wherein

the drilling parameters of the drilling system comprise

a hook load of a hook of the drilling system;

a pump rate of a pump of the drilling system; and

a torque value of a top drive of the drilling system; and

the model generates as an output at least one of

a weight on a drill bit (WOB) of the drilling system;

a rotation rate of the drill bit; and

a flow rate of drilling fluid through the drilling system.

12. The apparatus of claim 11, wherein the set of instructions that cause the processor to generate the control signal to the drilling system further cause the processor to generate a control signal corresponding to at least one of the drilling parameters.

13. The apparatus of claim 11, wherein the set of instructions that cause the processor to determine the maximum rate of penetration for the drilling system further cause the processor to determine values of the WOB, rotation rate, and flow rate that correspond to the maximum rate of penetration.

14. The apparatus of claim 9, wherein the set of instructions that cause the processor to calculate the control policy for the drilling system further cause the processor to

track the differences between the second set of values and the values of the WOB, rotation rate, and flow rate that correspond to the maximum rate of penetration using a cost function;

calculate a value function corresponding to the lowest average output of the cost function;

calculate a control input for each of the state of the drilling system using the value function; and

generate a look-up table containing the control inputs and the states of the drilling system.

15. The apparatus of claim 14, wherein the set of instructions that cause the processor to generate the control signal to the drilling system based, at least in part, on the control policy further cause the processor to

generate a real-time estimation of a state of the drilling system;

select a control input from the look-up table that corresponds to the estimated state;

generate the control signal for the drilling system using the control input.

16. The apparatus of claim 9, wherein the set of instructions further cause the processor to

receive a second set of downhole measurements;

generate a second model of the drilling system based, at least in part, on the second set of downhole measurements;

calculate a second control policy based, at least in part, on the second model; and

generate a second control signal to the drilling system based, at least in part, on the second control policy.