Collection and validation of data from visual displays

• US 10,140,741 B2
• Filed: 12/14/2015
• Issued: 11/27/2018
• Est. Priority Date: 12/14/2015
• Status: Active Grant

• Current Assignee: Baker Hughes Holdings LLC (Baker Hughes Company)
• Original Assignee: General Electric Company
• Inventors: Leigh, Nigel Trevor, Taylor, Brett Ralph, Craft, Robert Steven, Post, Stephen James
• Primary Examiner(s): Patel, Shivang
• Application Number:
  US14/967,913
• Publication Number:
  US 20170169593A1
• Time in Patent Office: 1,079 Days
• Field of Search: 345633
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30121 CRT, LCD or plasma display
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 2201/02 Recognising information on displays, dials, clocks
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments

Abstract

A mobile computing device for providing an indication of process readings of a machine includes a processor configured to receive an image of a process indicator wherein capture of the image is initiated using the mobile computing device proximate the machine. The at least one processor is also configured to determine a template including a template image of the process indicator and template components of the process indicator. The at least one processor is further configured to display the received image and the determined template image such that they overlay each other and align the received image with the determined template image. The at least one processor is also configured to determine a position of a value indicator with respect to a scale of the process indicator, transform the determined position to a process indicator value, and store the process indicator value in the one or more memory devices.

Claims

1. A mobile handheld computing device for providing an indication of process readings of a machine, the mobile handheld computing device comprising:

a memory device;

a display device;

a camera; and

at least one processor coupled to a non-transitory storage medium storing computer readable instructions, each of the memory device, the display device, the camera, and the at least one processor being integrated within said mobile handheld computing device, the processor being configured to, when executing the computer readable instructions;

receive an image of a process indicator from the camera, wherein receiving the image is initiated using the mobile handheld computing device after the mobile handheld computing device is carried to a location proximate the machine;

determine a template from a plurality of stored templates, each template associated with a process indicator of a plurality of process indicators, the determined template associated with the process indicator captured in the received image at the time the image was captured, the template retrieved from the memory device, the template including a template image of the process indicator and template components of the process indicator;

display the received image and the determined template image such that one of the received image and the determined template image overlay the other of the received image and the determined template image;

align visual elements in the received image and surrounding environment with corresponding framing elements of the determined template image;

determine at least one of a position of a value indicator shown in the received image with respect to a scale of the process indicator shown in the received image and characters of a digital process indicator;

transform the at least one of the determined position and the determined characters to a process value; and

store the process value in the memory device.

2. The mobile handheld computing device of claim 1, wherein the at least one processor is further configured to characterize the process indicator using at least one of a shape of a bezel of the process indicator, a shape of the scale of the process indicator, labels proximate the process indicator, and a color scheme of the process indicator.

3. The mobile handheld computing device of claim 2, wherein the at least one processor is further configured to optically recognize the labels.

4. The mobile handheld computing device of claim 3, wherein the at least one processor is further configured to identify the process indicator using at least one of the characterization and the labels.

5. The mobile handheld computing device of claim 1, wherein the at least one processor is further configured to:

extract one or more features of the process indicator in the received image; and

retrieve a template image from a plurality of predetermined template images stored in the memory device, the retrieved template image including predetermined features that match the extracted one or more features of the process indicator in the received image.

6. The mobile handheld computing device of claim 1, wherein the at least one processor is further configured to:

correlate the determined position of the value indicator to a template position on the determined template image; and

determine a process indicator displayed value based on the correlated position.

7. The mobile handheld computing device of claim 1, wherein the template components of the process indicator include at least one of a scale range, a scale minimum value, a scale maximum value, scale increments, and thresholds associated with the process indicator.

8. The mobile handheld computing device of claim 1, wherein the at least one processor is further configured to:

determine that alignment of the received image with the determined template image has been achieved; and

automatically collect the process value.

9. A computer-based method for providing an indication of process readings of a machine, the method implemented using a handheld mobile computing device coupled to a user interface, a camera, and a memory device, the method comprising:

receiving, by the handheld mobile computing device, an image of a process indicator captured from the camera, wherein receiving the image is initiated using the handheld mobile computing device while the handheld mobile computing device is proximate the machine, each of the memory device, a display device, the camera, and at least one processor being integrated within the handheld mobile computing device;

determining, by the handheld mobile computing device, a template associated with the process indicator from a plurality of stored templates, the determined template including a template image of the process indicator captured from the camera and template components of the process indicator;

displaying the received image and the determined template image overlaid together;

aligning visual elements in the received image and surrounding environment with corresponding framing elements of the determined template image;

determining a position of a value indicator with respect to a scale of the process indicator;

transforming the determined position to a process indicator value; and

storing the process indicator value in the memory device.

10. The method of claim 9, wherein the method further comprises characterizing the process indicator using at least one of a shape of a bezel of the process indicator, a shape of the scale of the process indicator, labels proximate the process indicator, and a color scheme of the process indicator.

11. The method of claim 10, wherein the method further comprises optically recognizing the labels.

12. The method of claim 11, wherein the method further comprises identifying the process indicator using at least one of the characterization and the labels.

13. The method of claim 9, wherein the method further comprises:

extracting one or more features of the process indicator in the received image; and

retrieving a template from a plurality of predetermined templates stored in the memory device, the retrieved template including predetermined features that match the extracted one or more features of the process indicator in the received image.

14. The method of claim 9, wherein the method further comprises:

correlating the determined position of the value indicator to a template position on the determined template image; and

determining a process indicator displayed value based on the correlated position.

15. The method of claim 9, further comprising storing, by the handheld mobile computing device, template components of the process indicator that include at least one of a scale range, a scale minimum value, a scale maximum value, scale increments, and thresholds associated with the process indicator.

16. One or more non-transitory computer-readable storage media having computer-executable instructions embodied thereon, wherein when executed by at least one processor of a handheld mobile computing device, the computer-executable instructions cause the processor to:

receive an image of a process indicator from a camera of the handheld mobile computing device, wherein receiving the image is initiated using the handheld mobile computing device while the handheld mobile computing device is proximate the machine, each of a memory device, a display device, the camera, and the at least one processor being integrated within said handheld mobile computing device;

determine a template associated with the process indicator, the determined template including a template image of the process indicator and template components of the process indicator;

display the received image and the determined template image overlaid together;

align visual elements in the received image and surrounding environment with corresponding framing elements of the determined template image;

determine a position of a value indicator with respect to a scale of the process indicator;

transform the determined position to a process indicator value; and

store the process indicator value in the memory device.

17. The computer-readable storage media of claim 16, wherein the computer-executable instructions further cause the processor to characterize the process indicator using at least one of a shape of a bezel of the process indicator, a shape of the scale of the process indicator, labels proximate the process indicator, and a color scheme of the process indicator.

18. The computer-readable storage media of claim 17, wherein the computer-executable instructions further cause the processor to optically recognize the labels.

19. The computer-readable storage media of claim 17, wherein the computer-executable instructions further cause the processor to identify the process indicator using at least one of the characterization and the labels.

20. The computer-readable storage media of claim 16, wherein the computer-executable instructions further cause the processor to:

extract one or more features of the process indicator in the received image; and

retrieve a template from a plurality of predetermined templates stored in the memory device, the retrieved template including predetermined features that match the extracted one or more features of the process indicator in the received image.

21. The computer-readable storage media of claim 16, wherein the computer-executable instructions further cause the processor to:

correlate the determined position of the value indicator to a template position on the determined template image; and

determine a process indicator displayed value based on the correlated position.