Initialization of radial base function neural network nodes for reinforcement learning incremental control system

• US 11,879,656 B2
• Filed: 04/04/2018
• Issued: 01/23/2024
• Est. Priority Date: 04/04/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hurley, Steven, Wilson, Brody, McCleary, Donald
• Primary Examiner(s): Fennema, Robert E
• Assistant Examiner(s): Tran, Vi N
• Application Number:
  US15/944,936
• Publication Number:
  US 20190309979A1
• Time in Patent Office: 2,120 Days
• CPC Class Codes: F24F 11/49 ensuring correct operation, e.g. by trial operation or configuration checks
• CPC Class Codes: F24F 11/58 using Internet communication
• CPC Class Codes: F24F 11/64 using pre-stored data
• CPC Class Codes: F24F 11/65 for selecting an operating mode
• CPC Class Codes: F24F 11/67 Switching between heating and cooling modes
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

According to one or more embodiments of the present invention, a computer-implemented method for adjusting a process variable using a closed loop system includes initializing a radial basis function neural network (RBF network) using a maximum error (e_max), a maximum first order change in error (Δe_max), a maximum second order change in error (Δ²e_max), and a maximum output increment (Δo_max), associated with the closed loop system being controlled. The method further includes inputting, to the RBF network, input values including an error, a first order change in error, and a second order change in error. The method further includes computing, by the RBF network, control parameters based on the input values, and computing, by the RBF network, an incremental change in the process variable based on the control parameters. The method further includes adjusting, by a controller, an output device to change the process variable by the incremental change.

Claims

1. A computer-implemented method for adjusting a process variable using a closed loop system, the method comprising:

initializing a radial basis function neural network (RBF network), the initialization using a maximum error (e_max), a maximum first order change in error (Δ

e_max), a maximum second order change in error (Δ

²e_max), and a maximum output increment (Δ

o_max), associated with the closed loop system being controlled, and wherein the initialization comprises setting center vectors (μ

) of the RBF network using the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

2. The computer-implemented method of claim 1, wherein the initialization comprises setting scalar widths (σ

) of the RBF network using mean of the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

σ

=∥

[e_max Δ

e_max Δ

²e_max]∥

.

3. The computer-implemented method of claim 1, wherein the initialization further comprises initializing an RBF network weight matrix of the RBF network by scaling an initial weight vector (ω

′

) so that an input vector including the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) multiplied by a predetermined fraction generates the maximum output increment (Δ

o_max).

4. The computer-implemented method of claim 1, wherein the RBF network computes an output vector as K(t)=θ

^Tϕ

(t), where θ

is an RBF network weight matrix, and ϕ

is an activation function of the RBF network.

5. The computer-implemented method of claim 4, wherein the incremental change is computed as Δ

o(t)=K(t)·

x(t), where x comprises the input values.

6. The computer-implemented method of claim 4, wherein the RBF network weight matrix is updated using an actor critic reinforced learning model.

7. A control system for controlling a process variable of a closed loop system, the control system comprising:

one or more sensors that measure the process variable;

an output device that changes the process variable based on an input command; and

a controller that generates and sends the input command to the output device, generating of the input command comprises;

initializing a radial basis function neural network (RBF network), the initialization using a maximum error (e_max), a maximum first order change in error (Δ

e_max), a maximum second order change in error (Δ

²e_max), and a maximum output increment (Δ

o_max), associated with the closed loop system being controlled, wherein the initialization comprises setting center vectors (μ

) of the RBF network using the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

8. The control system of claim 7, wherein the initialization comprises setting scalar widths (σ

) of the RBF network using mean of the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

σ

=∥

[e_max Δ

e_max Δ

²e_max]∥

.

9. The control system of claim 7, wherein the initialization further comprises initializing an RBF network weight matrix of the RBF network by scaling an initial weight vector (ω

′

) so that an input vector including the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) multiplied by a predetermined fraction generates the maximum output increment (Δ

o_max).

10. The control system of claim 7, wherein the RBF network computes an output vector as K(t)=θ

^Tϕ

(t), where θ

is an RBF network weight matrix, and ϕ

is an activation function of the RBF network.

11. The control system of claim 10, wherein the incremental change is computed as Δ

o(t)=K(t)·

x(t), where x comprises the input values.

12. A non-transitory computer readable storage medium for generating an input command for adjusting a process variable of a closed loop system, the non-transitory computer readable storage medium comprising a memory device with computer executable instructions therein, the instructions when executed by a processing unit generate the input command, the generation of the input command comprising:

initializing a radial basis function neural network (RBF network), the initialization using a maximum error (e_max), a maximum first order change in error (Δ

e_max), a maximum second order change in error (Δ

²e_max), and a maximum output increment (Δ

o_max), associated with the closed loop system being controlled, wherein the initialization comprises setting center vectors (μ

) of the RBF network using the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

13. The non-transitory computer readable storage medium of claim 12, wherein the initialization comprises setting scalar widths (σ

) of the RBF network using mean of the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

σ

=∥

[e_max Δ

e_max Δ

²e_max]∥

.

14. The non-transitory computer readable storage medium of claim 12, wherein the initialization further comprises initializing an RBF network weight matrix of the RBF network by scaling an initial weight vector (ω

′

) so that an input vector including the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) multiplied by a predetermined fraction generates the maximum output increment (Δ

o_max).

15. The non-transitory computer readable storage medium of claim 12, wherein the RBF network computes an output vector as K(t)=θ

^Tϕ

(t), where θ

is an RBF network weight matrix, and ϕ

is an activation function of the RBF network.

16. The non-transitory computer readable storage medium of claim 15, wherein the incremental change is computed as Δ

o(t)=K(t)·

x(t), where x comprises the input values.

17. A controller configured to adjust a process variable of a closed loop system to a set point, the adjustment comprising:

initializing a radial basis function neural network (RBF network), the initialization using a maximum error (e_max), a maximum first order change in error (Δ

e_max), a maximum second order change in error (Δ

²e_max), and a maximum output increment (Δ

o_max), associated with the closed loop system being controlled, wherein the initialization comprises setting center vectors (μ

) of the RBF network using the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

18. The controller of claim 17, wherein the initialization comprises:

setting scalar widths (σ

) of the RBF network using mean of the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

σ

=∥

[e_max Δ

e_max Δ

²e_max]∥

.

19. A control system for controlling a heat ventilating and air conditioning (HVAC) system, the control system comprising:

a controller that generates and sends an input command to the HVAC system to adjust a temperature to a set point, generating and sending of the input command comprises;

initializing a radial basis function neural network (RBF network), the initialization using a maximum error (e_max), a maximum first order change in error (Δ

e_max), a maximum second order change in error (Δ

²e_max), and a maximum output increment (Δ

o_max), associated with the closed loop system being controlled, wherein the initialization comprises setting center vectors (μ

) of the RBF network using the maximum error (e_max), the maximum first order change in error (Δ

e_max), and the maximum second order change in error (Δ

²e_max) as;

20. The control system of claim 19, wherein the error is a difference between the set point of the HVAC system and a measured temperature, the first order change in the error is a difference between the error and a previous error, and the second order change in the error is a difference between the first order change in the error and a previous first order change in the error.