Augmented Reality Visualization for Pipe Inspection

• US 20180259486A1
• Filed: 03/07/2018
• Published: 09/13/2018
• Est. Priority Date: 03/07/2017
• Status: Active Grant

• Current Assignee: Charles Stark Draper Laboratory Incorporated
• Original Assignee: Charles Stark Draper Laboratory Incorporated
• Inventors: Babcock, IV, Philip S., Stillo, Michael, Ryan, Kimberly, Vincent, Emily, Sprehn, Kelly, Brill, Ryan M., Busillo, Gregory G., Sun, Fei
• Granted Patent: US 10,564,127 B2
• Time in Patent Office: Days
• CPC Class Codes: G01N 17/006 of metals
• CPC Class Codes: G01N 2021/8893 providing a video image and a processed signal for helping visual decision
• CPC Class Codes: G01N 27/82 for investigating the presence of flaws
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G05B 2219/35482 Eyephone, head-mounted 2-D or 3-D display, also voice and other control
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

A defect visualization system includes an augmented reality display system to display an image representing a defect, such as missing metal, in a ferromagnetic material when a user gazes at a portion of the ferromagnetic material hosting the defect, based on magnetic field data provide by a magnetometry system, thereby facilitating locating the defect and replacing magnetometers after the defect has been repaired, even if location references originally present on cladding material are lost or destroyed during the repair.

Claims

1. A system for visualizing defects in ferromagnetic material, the system comprising:

a source of magnetic field data, the magnetic field data characterizing location of the defect in the ferromagnetic material;

a display device configured to generate an image perceivable by a human; and

an augmented reality system coupled to receive the magnetic field data and cause display, by the display device, of an image representing the magnetic field data, such that the image registers, as viewed by the human, with the ferromagnetic material.

2. A system according to claim 1, wherein the magnetic field data characterizes location of metal missing from the ferromagnetic material.

3. A system according to claim 1, wherein the display device comprises a head-mounted display.

4. A system according to claim 3, wherein the display device comprises a stereoscopic vision display device.

5. A system according to claim 1, wherein the display device comprises a 3D projector.

6. A system according to claim 5, wherein the display device comprises a holographic projector.

7. A system according to claim 1, wherein the display device comprises a hand-held device comprising a display screen.

8. A system according to claim 7, wherein the display screen comprises a stereoscopic vision display screen.

9. A system according to claim 1, wherein the display device further comprises a camera, and the display device is configured to display an image captured by the camera, overlaid with the image representing the magnetic field data.

10. A system according to claim 1, wherein the image representing the magnetic field data comprises first icons representing magnetometers that detected a magnetic field proximate the ferromagnetic material and from which the magnetic field data was derived, such that the first icons register, as viewed by the human, with the ferromagnetic material.

11. A system according to claim 10, wherein the image representing the magnetic field data comprises a second icon, visually distinct from the first icons, representing a failed magnetometer.

12. A system according to claim 1, wherein the image representing the magnetic field data comprises historical information about the magnetic field data.

13. A system according to claim 12, wherein the historical information comprises a graph showing estimated thickness of the ferromagnetic material over time.

14. A system according to claim 1, wherein the image representing the magnetic field data comprises a graph representing relative strength of a magnetic field proximate the ferromagnetic material and from which the magnetic field data was derived, such that the graph registers, as viewed by the human, with the ferromagnetic material.

15. A system according to claim 1, wherein the image representing the magnetic field data comprises a graphical representation of the defect in the ferromagnetic material, such that the graphical representation registers, as viewed by the human, with the ferromagnetic material.

16. A system according to claim 1, wherein the image representing the magnetic field data comprises a defect depth map, such that the defect depth map registers, as viewed by the human, with the ferromagnetic material.

17. A system according to claim 1, wherein the image representing the magnetic field data comprises one or more of:

a fault code, battery charge level information, a system serial number, sensor array identification information, system diagnostic information, geolocation information about the magnetic field data represented in the image and/or a failed sensor indicator.

18. A system according to claim 1, wherein the source of magnetic field data comprises a plurality of magnetometers disposed proximate a surface of the ferromagnetic material, wherein each magnetometer of the plurality of magnetometers is fixed in position, relative to the ferromagnetic material.

19. A system according to claim 1, wherein the source of magnetic field data comprises a plurality of magnetometers arranged in an array, the array being positionable at a standoff distance from the ferromagnetic material and translatable along an axis to a plurality of scan positions along the ferromagnetic material.

20. A system according to claim 19, wherein the image representing the magnetic field data comprises an indication of a path taken by the array, such that the indication of the path registers, as viewed by the human, with the ferromagnetic material.

21. A method for visualizing defects in ferromagnetic material, the method comprising:

generating magnetic field data, the magnetic field data characterizing location of the defect in the ferromagnetic material;

providing a display device configured to generate an image perceivable by a human;

providing an augmented reality system coupled to receive the magnetic field data; and

displaying, by the display device, an image representing the magnetic field data, such that the image registers, as viewed by the human, with the ferromagnetic material.

22. A method according to claim 21, wherein the magnetic field data characterizes location of metal missing from the ferromagnetic material.

23. A method according to claim 21, wherein providing the display device comprises providing a head-mounted display.

24. A method according to claim 23, wherein providing the display device comprises providing a stereoscopic vision display device.

25. A method according to claim 21, wherein providing the display device comprises providing a 3D projector.

26. A method according to claim 25, wherein providing the display device comprises providing a holographic projector.

27. A method according to claim 21, wherein providing the display device comprises providing a hand-held device comprising a display screen.

28. A method according to claim 27, wherein providing the display screen comprises providing a stereoscopic vision display screen.

29. A method according to claim 21, further comprising:

providing a camera;

displaying, by the display device, an image captured by the camera; and

overlaying the image captured by the camera with the image representing the magnetic field data.

30. A method according to claim 21, wherein displaying the image representing the magnetic field data comprises displaying first icons representing magnetometers that detected a magnetic field proximate the ferromagnetic material and from which the magnetic field data was derived, such that the first icons register, as viewed by the human, with the ferromagnetic material.

31. A method according to claim 30, wherein displaying the image representing the magnetic field data comprises displaying a second icon, visually distinct from the first icons, representing a failed magnetometer.

32. A method according to claim 21, wherein displaying the image representing the magnetic field data comprises displaying historical information about the magnetic field data.

33. A method according to claim 32, wherein displaying the historical information comprises displaying a graph showing estimated thickness of the ferromagnetic material over time.

34. A method according to claim 21, wherein displaying the image representing the magnetic field data comprises displaying a graph representing relative strength of a magnetic field proximate the ferromagnetic material and from which the magnetic field data was derived, such that the graph registers, as viewed by the human, with the ferromagnetic material.

35. A method according to claim 21, wherein displaying the image representing the magnetic field data comprises displaying a graphical representation of the defect in the ferromagnetic material, such that the graphical representation registers, as viewed by the human, with the ferromagnetic material.

36. A method according to claim 21, wherein displaying the image representing the magnetic field data comprises displaying a defect depth map, such that the defect depth map registers, as viewed by the human, with the ferromagnetic material.

37. A method according to claim 21, wherein displaying the image representing the magnetic field data comprises displaying one or more of:

a fault code, battery charge level information, system serial number, sensor array identification information, system diagnostic information, geolocation information about the magnetic field data represented in the image and/or a failed sensor indicator.

38. A method according to claim 21, wherein generating the magnetic field data comprises providing a plurality of magnetometers disposed proximate a surface of the ferromagnetic material, wherein each magnetometer of the plurality of magnetometers is fixed in position, relative to the ferromagnetic material.

39. A method according to claim 21, wherein generating the magnetic field data comprises:

providing a plurality of magnetometers arranged in an array;

positioning the array at a standoff distance from the ferromagnetic material; and

translating the array along an axis to a plurality of scan positions along the ferromagnetic material.

40. A method according to claim 39, wherein displaying the image representing the magnetic field data comprises displaying an indication of a path taken by the array, such that the indication of the path registers, as viewed by the human, with the ferromagnetic material.

41. A computer program product for visualizing defects in ferromagnetic material, the computer program product comprising a non-transitory computer-readable medium having computer readable program code stored thereon that, when executed by a processor, establishes processes, the processes comprising:

a process that generates magnetic field data, the magnetic field data characterizing location of the defect in the ferromagnetic material;

a process driving a display device configured to generate an image perceivable by a human; and

an augmented reality process coupled to receive the magnetic field data and cause display, by the display device, of an image representing the magnetic field data, such that the image registers, as viewed by the human, with the ferromagnetic material.

42. A computer program product according to claim 41, wherein the magnetic field data characterizes location of metal missing from the ferromagnetic material.