Monitor and control systems and methods for occupant safety and energy efficiency of structures

• US 9,948,872 B2
• Filed: 05/24/2013
• Issued: 04/17/2018
• Est. Priority Date: 03/02/2009
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Frank, Jeffrey D., Boulanger, Pierre, Jepson, Shawn, Richardson, Patrick B., Fairfield, Nile E., Hgasten, Nicholas, Hoelter, Theodore R., Strandemar, Katrin
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Uhl, Lindsay
• Application Number:
  US13/902,115
• Publication Number:
  US 20130321637A1
• Time in Patent Office: 1,789 Days
• Field of Search: None
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 25/677 for reducing the column or line fixed pattern noise
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Various systems and methods are disclosed for monitoring and controlling using small infrared imaging modules to enhance occupant safety and energy efficiency of buildings and structures. In one example, thermal images captured by infrared imaging modules may be analyzed to detect presence of persons, identify and classify power-consuming objects, and monitor environmental conditions. Based on the processed thermal images, various power-consuming objects (e.g., an HVAC system, lighting, a water heater, and other appliances) may be controlled to increase energy efficiency. In another example, thermal images captured by infrared imaging modules may be analyzed to detect various hazardous conditions, such as a combustible gas leak, a CO gas leak, a water leak, fire, smoke, and, an electrical hotspot. If such hazardous conditions are detected, an appropriate warning may be generated and/or various objects may be controlled to remedy the conditions.

Claims

1. A system comprising:

one or more infrared imaging modules comprising a focal plane array (FPA) configured to;

capture thermal images of an area, and

capture a thermal image frame that comprises thermal image data and noise introduced by the infrared imaging module;

a processor configured to;

determine, for each row of the thermal image frame, a corresponding row fixed-pattern noise (FPN) correction term, and for each column of the thermal image frame, a corresponding column FPN correction term;

apply the row and the column FPN correction terms to the thermal image frame to provide a corrected thermal image frame;

process the corrected image frame to determine a plurality of non-uniform correction (NUC) terms to reduce a portion of the noise comprising spatially uncorrelated FPN,

apply the NUC terms to the thermal images,

perform an analysis of the thermal images,

determine a presence of one or more persons present in the area based on the analysis of the thermal images,

identify one or more objects in the area,

determine respective power usage states associated with the one or more objects in the area based on the analysis of the thermal images, and

generate control signals to control future power usage associated with the one or more objects in response to the presence of one or more persons present in the area and the determined power usage states respectively associated with the one or more objects; and

a network interface component (NIC) configured to transmit the control signals to control the future power usage of the one or more objects.

2. The system of claim 1, wherein:

the one or more objects include at least one of a heater, an air conditioner, a heating ventilation air conditioning (HVAC) unit, a water heater, an oven, a lamp, or a lighting fixture; and

the one or more infrared imaging modules are radiometrically calibrated to generate the thermal images that is radiometrically normalized to a temperature scale.

3. The system of claim 1, wherein the processor is further configured to:

identify a hazardous condition in the area and to determine a source of the hazardous condition based on the analysis of the thermal images;

generate a control signal to remedy the hazardous condition if the source is controllable; and

generate a warning if the hazardous condition is not capable of being remedied.

4. The system of claim 3, wherein the identified hazardous condition includes at least one of a carbon monoxide (CO) gas leak, a combustible gas leak, fire, smoke, an electrical hot spot, a water leak, or flood.

5. The system of claim 1, wherein the processor is further configured to:

determine one or more environmental conditions including a humidity level of the area; and

generate the control signals in further response to the one or more determined environmental conditions of the area.

6. The system of claim 1, wherein the processor is further configured to identify respective object types of the one or more objects based on the analysis of the thermal images, and to generate the control signals in further response to the respective object types of the one or more objects.

7. The system of claim 1, wherein the processor is further configured determine energy usage associated with the one or more objects.

8. The system of claim 7, wherein the processor is further configured to:

perform an analysis of the energy usage associated with the one or more objects; and

make suggestions to increase energy efficiency of the one or more objects.

9. The system of claim 1, wherein:

the thermal images are unblurred thermal images of the area; and

the thermal image frame comprises an intentionally blurred thermal image frame that comprises thermal image data associated with the area and the noise introduced by the infrared imaging module.

19. The system of claim 1, wherein the processor is further configured to:

determine a number and physical statuses of persons present in the area based on the analysis of the thermal images, the physical statuses including postures and/or durations of motionlessness of persons present in the area determined based on the analysis of the thermal images;

determine that the persons present in the area are asleep or resting based on the postures and/or the duration of motionlessness of the persons; and

generate control signals to reduce the future power usage associated with the one or more objects in response to determining that the persons present in the area are asleep or resting.

10. A method comprising:

capturing, at a focal plane array of an infrared imaging module, thermal images of an area;

capturing a thermal image frame that comprises thermal image data and noise introduced by the infrared imaging module;

determining, for each row of the thermal image frame, a corresponding row fixed-pattern noise (FPN) correction term, and for each column of the thermal image frame, a corresponding column FPN correction term;

applying the row and the column FPN correction terms to the thermal image frame to provide a corrected thermal image frame;

processing the corrected image frame to determine a plurality of non-uniform correction (NUC) terms to reduce a portion of the noise comprising spatially uncorrelated FPN,

applying the NUC terms to the thermal images;

performing an analysis of the thermal images;

determining a presence of one or more persons present in the area based on the analysis of the thermal images;

identifying one or more objects in the area;

determining respective power usage states associated with the one or more objects in the area based on the analysis of the thermal images;

generating control signals to control future power usage associated with the one or more objects in response to the presence of one or more persons present in the area and the determined power usage states respectively with the one or more objects; and

communicating the control signals to the one or more objects over a network.

11. The method of claim 10, further comprising radiometrically normalizing the thermal images to a temperature scale, wherein the one or more objects include at least one of a heater, an air conditioner, a heating ventilation air conditioning (HVAC) unit, a water heater, an oven, a lamp, or a lighting fixture.

12. The method of claim 10, wherein:

identifying a hazardous condition in the area and determining a source of the hazardous condition based on the analysis of the thermal images; and

said generating of the control signals comprises;

generating a control signal to the source to remedy the hazardous condition if the source is controllable; and

generating a warning if the hazardous condition is not capable of being remedied.

13. The method of claim 12, wherein the identified hazardous condition includes at least one of a carbon monoxide (CO) gas leak, a combustible gas leak, fire, smoke, an electrical hot spot, a water leak, or flood.

14. The method of claim 10, wherein:

determining one or more environmental conditions including a humidity level of the area; and

said generating of the control signals is in further response to the one or more determined environmental conditions of the area.

15. The method of claim 10, wherein:

the method further comprises identifying respective object types of the one or more objects based on the analysis of the thermal images; and

said generating of the control signals is in further response to the respective object types of the one or more object types.

16. The method of claim 10, further comprising determining energy usage associated with the one or more objects.

17. The method of claim 16, further comprising:

performing an analysis of the energy usage associated with the one or more objects; and

making suggestions to increase energy efficiency of the one or more objects.

18. The method of claim 10, wherein:

the thermal images are unblurred thermal images; and

the thermal image frame comprises an intentionally blurred thermal image frame that comprises thermal image data associated with the external environment and the noise introduced by the infrared imaging module.

20. The method of claim 10, further comprising:

determining a number and physical statuses of persons present in the area based on the analysis of the thermal images, the physical statuses including postures and/or durations of motionlessness of persons present in the area determined based on the analysis of the thermal images;

determining that the persons present in the area are asleep or resting based on the postures and/or the duration of motionlessness of the persons; and

generating control signals to reduce the future power usage associated with the one or more objects in response to determining that the persons present in the area are asleep or resting.