Visible audiovisual annotation of infrared images using a separate wireless mobile device

• US 10,337,962 B2
• Filed: 03/14/2014
• Issued: 07/02/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Fluke Corporation (Fortive Corp.)
• Original Assignee: Fluke Corporation (Fortive Corp.)
• Inventors: Stuart, Michael D., Schlichting, Jordan, Neeley, John
• Primary Examiner(s): Messmore, Jonathan R
• Application Number:
  US14/214,082
• Publication Number:
  US 20140267765A1
• Time in Patent Office: 1,936 Days
• Field of Search: 348164
• CPC Class Codes: G01D 1/00 Measuring arrangements giving results other than momentary value of variable, of general application G01D3/00 takes precedence; in tariff metering apparatus G01D4/00; transducers not specially adapted for a specific variable G01D5/00
• CPC Class Codes: G01D 7/00 Indicating measured values
• CPC Class Codes: G01D 7/02 Indicating value of two or more variables simultaneously
• CPC Class Codes: G01D 7/08 using a common indicating element for two or more variables
• CPC Class Codes: G01D 9/00 Recording measured values
• CPC Class Codes: G01M 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G06F 15/00 Digital computers in general details G06F1/00 – G06F13/00; Data processing equipment in general
• CPC Class Codes: G06Q 20/085 involving remote charge determination or related payment systems
• CPC Class Codes: G06Q 20/0855 involving a third party
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G08B 5/00 Visible signalling systems, e.g. personal calling systems, remote indication of seats occupied
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G16H 10/40 for data related to laboratory analysis, e.g. patient specimen analysis
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04Q 2209/40 using a wireless architecture
• CPC Class Codes: H04Q 2209/47 using RFID associated with sensors
• CPC Class Codes: H04Q 2209/50 using a mobile data collecting device, e.g. walk by or drive by
• CPC Class Codes: H04Q 2209/86 Performing a diagnostic of the sensing device
• CPC Class Codes: H04Q 2209/883 : where the sensing device enters an active or inactive mode
• CPC Class Codes: H04Q 9/00 : Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Abstract

A smart phone user captures video and still images of a motor and makes a sound recording of his observations. The images and recordings are tagged with a date/time stamp, a serial number, bar code, matrix code, RFID code, or geolocation and sent wirelessly to an infrared camera. The infrared camera has a collocation application that reads the tag and creates a folder which holds the infrared image and the auxiliary information from the smartphone related to an asset. The collocation program also adds icons to the infrared image. By operating the user interface (not shown) of the infrared camera, the user may select an icon to view the images and listen to the sound recording.

Claims

1. A method for annotating an infrared image, comprising:

storing in a folder an infrared image of an asset captured by an infrared device that is not able to capture visible light still and video images and sound recordings, wherein the infrared image is stored in the folder with one or more tags associated with the asset;

receiving auxiliary information from a non-infrared device that is separate from the infrared device, wherein the auxiliary information has at least one tag corresponding to a tag associated with the asset, and wherein the auxiliary information is at least one of a visible light still or video image, or a sound recording;

operating a collocation program that sorts the auxiliary information received from the separate non-infrared device and, based on the at least one tag, stores the auxiliary information in the same folder with the infrared image;

generating an annotated infrared image by adding the infrared image one or more icons representative of types of the auxiliary information stored in the folder, wherein the one or more icons are symbols representative of still images, sound recordings, and video images, respectively;

storing the annotated infrared image; and

displaying to a user the annotated infrared image of the asset annotated with the one or more icons representative of the types of the auxiliary information stored in the folder.

2. The method of claim 1, further comprising wirelessly receiving the auxiliary information from the non-infrared device.

3. The method of claim 1, wherein each still or video image or sound recording has at least one tag corresponding to a tag associated with the asset.

4. The method of claim 1, further comprising receiving user-selection of an icon in the annotated infrared image, the icon representing a type of auxiliary information stored in the folder, and in response to said user-selection of the icon, further providing to the user auxiliary information represented by the selected icon.

5. The method of claim 1, wherein a tag comprises information representative of an identity of the asset.

6. The method of claim 1, wherein a tag comprises a coded image.

7. The method of claim 6, wherein the tag comprises a serial number, a barcode, or a matrix code.

8. The method of claim 1, wherein a tag comprises a date/time stamp of when the infrared image or the still or video image or sound recording was captured.

9. The method of claim 1, wherein a tag comprises a geolocation code corresponding to a geographic location of the asset.

10. The method of claim 1, wherein the icons are representative of one or more of a group consisting of still images, video images, or sound recordings stored as auxiliary information in the folder.

11. The method of claim 1, further comprising capturing one or more of a group comprising still images, video images, and sound recordings, wherein each still or video image or sound recording is stored with at least one tag corresponding to a tag associated with the asset.

12. The method of claim 1, further comprising capturing the infrared image of the asset with an infrared camera.

13. The method of claim 1, further comprising capturing a still image, a video image, or a sound recording with a smartphone and wirelessly transmitting the captured still or video images or sound recording for storage as auxiliary information in the folder holding the infrared image.

14. The method of claim 1, further comprising storing the annotated infrared image in an infrared camera.

15. The method of claim 1, further comprising storing the annotated infrared image in a smartphone.

16. The method of claim 1, further comprising storing the annotated infrared image in a computer.

17. The method of claim 16, wherein the computer comprises a personal computer or a tablet computer.

18. The method of claim 1, further comprising storing the annotated infrared image in a computer cloud storage.

19. A system for annotating an infrared image, comprising:

a storage apparatus that stores in a folder an infrared image of an asset captured by an infrared camera, wherein the infrared image is stored in the folder with one or more tags associated with the asset;

a non-infrared device that captures auxiliary information and at least one tag associated with the asset, wherein the auxiliary information is at least one of a still image, a sound recording, or a video image, and wherein the non-infrared device is separate from the infrared camera and the storage apparatus and wirelessly transmits the auxiliary information and the at least one tag to the storage apparatus;

a collocation program that sorts the auxiliary information into the folder corresponding to the at least one tag associated with the asset, and annotates the infrared image in the folder by adding to the infrared image one or more icons representative of the types of the auxiliary information stored in the folder, thus generating an annotated infrared image, wherein the one or more icons are symbols representative of still images, sound recordings, and video images, respectively;

a processor in the storage apparatus that operates the collocation program to store the auxiliary information in the folder and to add the one or more icons to the infrared image in the folder to generate the annotated infrared image; and

a display that displays the annotated infrared image of the asset.

20. The system of claim 19, wherein a tag comprises information representative of an identity of the asset.

21. The system of claim 19, wherein a tag comprises a coded image.

22. The system of claim 21, wherein the tag comprises a serial number, a barcode, or a matrix code.

23. The system of claim 19, wherein a tag comprises a date/time stamp of when the infrared image was captured.

24. The system of claim 19, wherein a tag comprises a geolocation code corresponding to a geographic location of the asset.

25. The system of claim 19, wherein the non-infrared device that captures the auxiliary information comprises one or more of a group consisting of a still image camera, a video camera, and a sound recorder that captures one or more of a group comprising still images, video images, and sound recordings associated with the asset.

26. The system of claim 19, wherein the storage apparatus comprises an infrared camera that captures the infrared image of the asset.

27. The system of claim 19, wherein the storage apparatus is one or more of a group consisting of a smartphone, a computer cloud storage, a personal computer, a tablet computer, a video camera, a digital camera, and a digital recorder.

28. An apparatus for annotating an infrared image comprising:

an input that receives wirelessly transmitted auxiliary information from a non-infrared device that is separate from the apparatus, wherein the auxiliary information is at least one of a still image, a sound recording, or a video image, and wherein the auxiliary information is associated with one or more tags;

a memory that stores in a folder an infrared image of an asset, the infrared image being captured by an infrared camera that is not able to capture said auxiliary information and is housed separate from the non-infrared device, wherein the infrared image is associated with one or more tags;

a collocation program that sorts the auxiliary information into folders according to the one or more tags associated with the auxiliary information, including sorting auxiliary information having one or more tags that correspond to the one or more tags associated with the asset into the folder storing the infrared image of the asset, wherein the collocation program further annotates the infrared image stored in the folder by adding to the infrared image one or more icons representative of the types of the auxiliary information stored in the folder, thus generating an annotated infrared image, wherein the one or more icons are symbols representative of still images, sound recordings, and video images, respectively;

a processor that operates the collocation program to store the auxiliary information in folders having infrared images with tags corresponding to the one or more tags of the auxiliary information, and to add the one or more icons to the infrared images representative of the types of auxiliary information stored in the respective folders to generate annotated infrared images; and

a display that displays the annotated infrared image of the asset.

29. The apparatus of claim 28, wherein said apparatus is an infrared camera.

30. The apparatus of claim 28, wherein a tag comprises information representative of an identity of the asset.

31. The apparatus of claim 28, wherein a tag comprises a coded image.

32. The apparatus of claim 31, wherein the tag comprises a serial number, a barcode, or a matrix code.

33. The apparatus of claim 28, wherein a tag comprises a date/time stamp of when the infrared image was captured.

34. The apparatus of claim 28, wherein a tag comprises a geolocation code corresponding to a geographic location of the asset.

35. The apparatus of claim 28, wherein the auxiliary information is wirelessly received from one or more non-infrared devices of a group consisting of a still image camera, a video camera, and a sound recorder that captures one or more of a group comprising still images, video images, and sound recordings.

36. The apparatus of claim 28, wherein the apparatus is one or more of a group consisting of a smartphone, a computer cloud storage, a personal computer, a tablet computer, a video camera, a digital camera, and a digital recorder.