AUTOMATED CONTROL AND PARALLEL LEARNING HVAC APPARATUSES, METHODS AND SYSTEMS

• US 20160223218A1
• Filed: 01/29/2016
• Published: 08/04/2016
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Schneider Electric IT Corporation (Schneider Electric SE)
• Original Assignee: Schneider Electric IT Corporation (Schneider Electric SE)
• Inventors: Barrett, Enda
• Granted Patent: US 10,465,931 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: F24F 11/30 for purposes related to the operation of the system, e.g. for safety or monitoring
• CPC Class Codes: F24F 11/46 Improving electric energy efficiency or saving
• CPC Class Codes: F24F 11/52 Indication arrangements, e.g. displays
• CPC Class Codes: F24F 11/54 using one central controller connected to several sub-controllers
• CPC Class Codes: F24F 11/56 Remote control
• CPC Class Codes: F24F 11/58 using Internet communication
• CPC Class Codes: F24F 11/62 characterised by the type of control or by internal processing, e.g. using fuzzy logic, adaptive control or estimation of values
• CPC Class Codes: F24F 11/64 using pre-stored data
• CPC Class Codes: F24F 11/65 for selecting an operating mode
• CPC Class Codes: F24F 2120/20 Feedback from users
• CPC Class Codes: F24F 2140/60 Energy consumption

Abstract

The AUTOMATED CONTROL AND PARALLEL LEARNING HVAC APPARATUSES, METHODS AND SYSTEMS (“ACPLHVAC”) updates real time value function estimates through parallel and reinforcement learning, via ACPLHVAC components, by observing a defined state action space to maximize user Quality of Experience (QoE) and minimize associated energy required with regulating environmental spaces.

Claims

1. A processor-implemented method, comprising:

establishing a connection between a first thermostat and a thermostat network, the thermostat network including a plurality of thermostats;

initializing the first thermostat;

receiving sensor data associated with the first thermostat;

receiving user feedback associated with the first thermostat;

receiving connected unit aggregated learning probability distributions;

selecting an HVAC action from a predefined set of available HVAC actions from a transition policy based on internal and external transition probabilities;

entering a new state following implementation of the selected HVAC action;

receiving a reward in the form of a positive or negative scalar associated with the selected HVAC action and new state;

updating an internal probability distribution;

determining divergence between internal probability distribution and external probability distributions; and

transmitting the internal probability distribution to the thermostat network if divergence is above a divergence threshold.

2. The method of claim 1, wherein the initialization is with arbitrary or pre-loaded state values.

3. The method of claim 1, wherein the plurality of thermostats are distributed within a specified geographic region.

4. The method of claim 1, wherein at least a portion of the plurality of thermostats are at least initially independently controlled.

5. A processor-implemented method, comprising:

initializing a thermostat to arbitrary or pre-loaded state values;

receiving multiple user feedback in the form of scalars denoting comfort;

aggregating user feedback into a scalar denoting mean user comfort;

selecting an HVAC action from a predefined set of available HVAC actions based on a transition policy informed by internal and external transition probabilities;

entering the next state following the selected HVAC action;

receiving a reward in the form of a positive or negative scalar associated with the selected HVAC action and new state as it approaches mean user comfort levels at minimized required energy; and

updating and exporting probability distributions.

6. A non-transitory computer readable medium having computer readable instructions stored thereon that, when executed by a processor of a computing device, cause the computing device to:

initialize to arbitrary or pre-loaded state values;

receive multi-user feedback in the form of scalars denoting comfort;

aggregate user feedback into a scalar denoting mean user comfort;

select an HVAC action from a predefined set of available HVAC actions based on a transition policy informed by internal and external transition probabilities;

enter the next state following the selected HVAC action;

receiving a positive or negative scalar associated with the selected HVAC action and new state to meet at minimized energy cost without exceeding threshold user comfort levels.