Systems and approaches for repeated thermal imaging determinations

• US 10,116,885 B2
• Filed: 04/05/2016
• Issued: 10/30/2018
• Est. Priority Date: 04/05/2015
• Status: Active Grant

• Current Assignee: ECB Consulting LLC
• Original Assignee: Hema Imaging LLC
• Inventors: Beall, Erik
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): Sechser, Jill
• Application Number:
  US15/091,317
• Publication Number:
  US 20160295208A1
• Time in Patent Office: 938 Days
• CPC Class Codes: G01J 2005/0077 Imaging
• CPC Class Codes: G06K 7/10297 arrangements for handling protocols designed for non-contact record carriers such as RFIDs NFCs, e.g. ISO/IEC 14443 and 18092 cryptographic protocols H04L9/00; network security protocols H04L63/00; real-time communication protocols in data switching networks H04L65/00; network protocols for data switching network services H04L67/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A thermal imaging system includes a thermal imaging sensor configured to capture thermal images of assets in an environment, a non-thermal imaging sensor coupled to the thermal sensor and being configured to capture non-thermal images of the assets, an image alignment system configured to adjust an orientation of a present thermal view of the particular asset and a present non-thermal view of the particular asset, a controller configured to control operation of the thermal imaging sensor, the non-thermal imaging sensor, and the image alignment system, and a remote computing device having a change detection system and being configured to receive the captured thermal and non-thermal images of the particular asset. The controller is configured to cause the sensors to capture thermal and non-thermal images. Upon receiving the captured thermal and non-thermal images, the change detection system is configured to detect a change in at least one characteristic of the asset.

Claims

1. A thermal imaging system, the system comprising;

a thermal imaging sensor configured to capture a plurality of thermal images containing thermal data of a plurality of assets in an environment;

a non-thermal imaging sensor coupled to the thermal sensor, the non-thermal imaging sensor configured to capture a plurality of non-thermal images of the plurality of assets;

an image alignment system;

an asset identification system configured to identify a particular asset from the plurality of assets, the particular asset being previously identified by being input into the asset identification system, the asset identification system identifying the particular asset by instructing a user to capture a unique identifier image associated with the particular asset and comparing the capture to a plurality of baseline unique identifier images;

a controller configured to control operation of the thermal imaging sensor, the non-thermal imaging sensor, and the image alignment system by;

(i) presenting at least one of a previously-captured baseline image or a template image of the particular asset to the user via a display such that the user may approximate an orientation of a present thermal view of the particular asset and a present non-thermal view of the particular asset to the at least one of the previously-captured baseline image or template image,

(ii) adjusting the orientation of a present thermal view of the particular asset and the present non-thermal view of the particular asset to match an orientation of at least one of the previously-captured baseline image or template image of the particular asset, and

(iii) causing the thermal imaging sensor to capture a thermal image of the present thermal view of the particular asset and the non-thermal imaging sensor to capture a non-thermal image of the present non-thermal view of the particular asset; and

a remote computing device configured to receive the captured thermal and non-thermal images of the particular asset, the remote computing device having a change detection system configured to detect a change in at least one characteristic of the particular asset using the thermal image captured by the thermal imaging sensor.

2. The thermal imaging system of claim 1, wherein the change detection system is configured to generate an image-based threshold using at least one thermal image captured by the thermal imaging sensor.

3. The thermal imaging system of claim 2, wherein the change detection system is configured to detect the change in the at least one characteristic of the particular asset by comparing the image-based threshold to at least one later-captured thermal image.

4. The thermal imaging system of claim 3, wherein the change detection system is further configured to generate a notification when the at least one later-captured thermal image falls outside of the image-based threshold.

5. The thermal imaging system of claim 4, wherein the change detection system is configured to use the thermal data to compute a temperature value of at least a portion of the particular asset.

6. The thermal imaging system of claim 5, wherein the temperature value comprises an average temperature value of all of the thermal data associated with the particular asset provided in the captured thermal image.

7. The thermal imaging system of claim 6, wherein the temperature value comprises an average temperature value of the thermal data associated with at least one pixel of the captured thermal image.

8. The thermal imaging system of claim 2, wherein change detection system is configured to adjust the image-based threshold.

9. The thermal imaging system of claim 1, wherein the change detection system is further configured to generate an image-based threshold using at least one non-thermal image captured by the non-thermal imaging sensor and detect a change in at least one characteristic of the particular asset using the non-thermal image.

10. The thermal imaging system of claim 9, wherein the change detection system is configured to use the non-thermal image to determine if a noticeable change in a surface appearance has occurred by examining at least one of the color and the luminance of the particular asset.

11. The thermal imaging system of claim 1, wherein the change detection system is configured to calculate a probability distribution of the thermal data and a standard deviation of the captured thermal image.

12. The thermal imaging system of claim 1, wherein the thermal data further comprises an ambient temperature value representing an ambient temperature of the captured thermal image.

13. The thermal imaging system of claim 12, wherein the change detection system adjusts for a difference in the ambient temperature value of the plurality of captured thermal images by subtracting the difference in the ambient temperature at each acquisition from each pixel of the captured thermal images.

14. The thermal imaging system of claim 12, wherein the change detection system adjusts for a difference in the ambient temperature value of the plurality of captured thermal images by subtracting a change in a background region of the thermal image previously identified to be a surrogate.

15. The thermal imaging system of claim 1, wherein the change detection system is configured to perform an empirical correction on the thermal data based on a change in the environment.

16. A method of repeatably acquiring thermal images in an environment, the method comprising:

identifying, via an asset identification system, at least one asset being previously identified, the asset identification system identifying the at least one asset by instructing a user to obtain a unique identifier image associated with the at least one asset and comparing the obtained unique identifier image to a plurality of baseline unique identifier images;

obtaining at least one present thermal view of the at least one asset using a thermal imaging sensor and at least one present non-thermal view of the at least one asset using a non-thermal imaging sensor;

presenting, via the image alignment system, at least one of a previously-captured baseline image or a template image of the at least one asset to a user via a display such that the user may approximate an orientation of the at least one present thermal view of the at least one asset and the present non-thermal view of the at least one asset to an orientation of the at least one of previously-captured baseline image or template image;

adjusting, via an image alignment system, the orientation of the at least one present thermal view of the at least one asset and the at least one present non-thermal view of the at least one asset to match the orientation of the at least one of previously-captured baseline image or template image of the at least one asset;

capturing at least one thermal image of the at least one present thermal view and at least one non-thermal image of the at least one present non-thermal view, the at least one thermal image having thermal image data;

transferring the at least one thermal image and the at least one non-thermal image to a processing device having a change detection system;

identifying, using the change detection system, at least one image-based threshold of the at least one thermal image; and

upon capturing a subsequent thermal image, comparing, via the change detection system, the at least one image-based threshold to a portion of the thermal image data of the subsequently captured thermal image.

17. The method of claim 16, wherein upon the portion of the thermal image data of the subsequently captured thermal image exceeding the at least one image-based threshold, the change detection system is configured to generate an alarm.

18. The method of claim 17, wherein the change detection system is further configured to cause the at least one image-based threshold and the subsequently captured thermal image to be displayed.

19. The method of claim 16, further comprising identifying, using the change detection system, at least one image-based threshold of the at least one non-thermal image, and upon capturing a subsequent non-thermal image, comparing, via the change detection system, the at least one image-based threshold to a portion of image data of the subsequently captured non-thermal image.

20. A thermal imaging device, the device comprising:

a housing;

a thermal imaging sensor at least partially arranged on an exterior surface of the housing and configured to capture a plurality of thermal images of a plurality of assets in an environment;

a non-thermal imaging sensor at least partially arranged on the exterior surface of the housing and coupled to the thermal sensor, the non-thermal imaging sensor configured to capture a plurality of non-thermal images of the plurality of assets;

an asset identification system configured to identify a particular asset from the plurality of assets, the particular asset being previously identified by being input into the asset identification system, the asset identification system identifying the particular asset by instructing a user to capture a unique identifier image associated with the particular asset and comparing the capture to a plurality of baseline unique identifier images;

an image alignment system configured to adjust an orientation of a present thermal view of the particular asset and a present non-thermal view of the particular asset to match an orientation of at least one of a previously-captured baseline image or a template image of the particular asset;

a communication module; and

a controller configured to control operation of the thermal imaging sensor, the non-thermal imaging sensor, the image alignment system, and the communication module;

wherein the controller is configured to (i) present at least one of a previously-captured baseline image or a template image of the particular asset to a user via a display such that the user may approximate an orientation of a present thermal view of the particular asset and a present non-thermal view of the particular asset to the at least one of the previously-captured baseline image or template image, (ii) cause the thermal imaging sensor to capture a thermal image of the present thermal view of the particular asset and the non-thermal imaging sensor to capture a non-thermal image of the present non-thermal view of the particular asset and iii cause the communication module to transmit the thermal image and the non-thermal image to a remote computing device, wherein upon receiving the plurality of captured thermal images and the plurality of captured non-thermal images, the remote computing device is configured to initiate a change detection system that detects a change in at least one characteristic of the particular asset using the plurality of captured thermal images and the plurality of captured non-thermal images.