Rapid Automated Infrared Thermography for Inspecting Large Composite Structures

• US 20170052070A1
• Filed: 08/17/2015
• Published: 02/23/2017
• Est. Priority Date: 08/17/2015
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Thompson, Jeffrey R., Marsh, Bobby J., Georgeson, Gary E.
• Granted Patent: US 9,645,012 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01D 5/347 using displacement encoding scales
• CPC Class Codes: G01J 2005/0077 Imaging
• CPC Class Codes: G01J 5/026 Control of working procedures of a pyrometer, other than calibration; Bandwidth calculation; Gain control
• CPC Class Codes: G01J 5/07 Arrangements for adjusting the solid angle of collected radiation, e.g. adjusting or orienting field of view, tracking position or encoding angular position optical collimating elements G01J5/0806
• CPC Class Codes: G01J 5/0846 having multiple detectors for performing different types of detection, e.g. using radiometry and reflectometry channels
• CPC Class Codes: G01J 5/10 using electric radiation detectors
• CPC Class Codes: G01J 5/48 Thermography; Techniques using wholly visual means
• CPC Class Codes: G01N 25/72 Investigating presence of flaws
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/56 : provided with illuminating means
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Systems and methods for infrared thermographic inspection of large-scale composite structures such as sections of an aircraft fuselage. Optical metrology is used to precisely locate the infrared images relative to a three-dimensional coordinate system of the composite structure. The optical metrology may comprise laser tracking or photogrammetry or both. In some embodiments, the optical metrology comprises laser tracking merged with photogrammetry. Once the infrared images have been precisely located relative to the coordinate system of the composite structure, structural data about the composite structure (e.g., thickness data) can be retrieved from a database containing a three-dimensional model of the composite structure. In the case of thermographic porosity measurements, the infrared imaging data can be correlated with thickness data to determine the porosity of the composite structure in the inspection area.

Claims

1. A method for infrared thermographic inspection of a composite structure comprising:

(a) moving an infrared camera to a first location whereat a field of view of the infrared camera encompasses a first inspection area of a surface of the composite structure;

(b) determining a first coordinate location of the field of view of the infrared camera in a coordinate system of the composite structure using optical metrology while the infrared camera is at the first location;

(c) activating at least one flash lamp to output light that illuminates at least portions of the first inspection area;

(d) activating the infrared camera to acquire first infrared imaging data while the field of view of the infrared camera encompasses at least the first inspection area;

(e) moving the infrared camera to a second location whereat the field of view of the infrared camera encompasses a second inspection area of the surface of the composite structure;

(f) determining a second coordinate location of the field of view of the infrared camera in the coordinate system of the composite structure using optical metrology while the infrared camera is at the second location;

(g) activating at least one flash lamp to output light that illuminates at least portions of the second inspection area;

(h) activating the infrared camera to acquire second infrared imaging data while the field of view of the infrared camera encompasses at least the second inspection area; and

(i) stitching the first and second infrared imaging data together based on at least the first and second coordinate locations of the field of view of the infrared camera in the coordinate system of the composite structure.

2. The method as recited in claim 1, wherein said optical metrology comprises laser tracking.

3. The method as recited in claim 2, wherein steps (b) and (f) collectively comprise:

placing optical targets on the composite structure;

directing respective pulses of light from a laser tracker toward the optical targets on the composite structure;

processing light returned from the optical targets on the composite structure to the laser tracker to determine first location data representing a coordinate location of the composite structure in a coordinate system of the laser tracker;

placing optical targets on a robot base that supports a robotic arm which supports the infrared camera;

directing respective pulses of light from the laser tracker toward the optical targets on the robot base;

processing light returned from the optical targets on the robot base to the laser tracker to determine second location data representing a coordinate location of the robot base in the coordinate system of the laser tracker;

encoding movements of the robotic arm; and

computing the first and second coordinate locations of the field of view of the infrared camera in the coordinate system of the composite structure based on at least the first and second location data and encoded movements of the robotic arm.

4. The method as recited in claim 2, wherein steps (b) and (f) collectively comprise:

placing optical targets on the composite structure;

directing respective pulses of light from one or more laser trackers toward the optical targets on the composite structure;

processing light returned from the optical targets on the composite structure to determine first location data representing a coordinate location of the composite structure in a coordinate system of the one or more laser trackers;

attaching respective sets of optical targets to first, second and third bars;

placing the first, second and third bars adjacent the composite structure in respective locations so that the first and second bars are separated by the first inspection area, and the second and third bars are separated by the second inspection area;

directing respective pulses of light from the one or more laser trackers toward the optical targets on the first and second bars during activations of the infrared camera in step (d);

directing respective pulses of light from the one or more laser trackers toward the optical targets on the second and third bars during activations of the infrared camera in step (h);

processing light returned from the optical targets on the first, second and third bars to determine second location data representing respective coordinate locations of the first, second and third bars in the coordinate system of the one or more laser trackers; and

computing the first and second coordinate locations of the field of view of the infrared camera in the coordinate system of the composite structure based on at least the first and second location data and the first and second infrared imaging data.

5. The method as recited in claim 4, wherein computing the first and second coordinate locations of the field of view of the infrared camera in the coordinate system of the composite structure comprises identifying portions of the first infrared imaging data which correspond to light returned from the optical targets on the first and second bars.

6. The method as recited in claim 1, wherein said optical metrology comprises photogrammetry.

7. A method for infrared thermographic inspection comprising:

(a) attaching a pair of photogrammetry cameras to an infrared camera to form a camera assembly;

(b) moving the camera assembly to a first location whereat respective fields of view of the photogrammetry cameras encompass a first inspection area of a surface of the composite structure;

(c) determining a first coordinate location of a field of view of the photogrammetry cameras in a coordinate system of the composite structure using optical metrology while the camera assembly is at the first location;

(d) activating at least one flash lamp to output light that illuminates at least portions of the first inspection area;

(e) activating the infrared camera to acquire first infrared imaging data while the field of view of the infrared camera encompasses at least the first inspection area;

(f) activating a projector to project a pattern of light onto the first inspection area while the camera assembly is at the first location;

(g) activating the photogrammetry cameras to acquire first photogrammetry data while the field of view of the photogrammetry cameras encompasses at least a portion of the projected pattern of light on the first inspection area;

(h) moving the camera assembly to a second location whereat the field of view of the photogrammetry cameras encompasses a second inspection area of the surface of the composite structure;

(i) determining a second coordinate location of the field of view of the photogrammetry cameras in the coordinate system of the composite structure using optical metrology while the camera assembly is at the second location;

(j) activating at least one flash lamp to output light that illuminates at least portions of the second inspection area;

(j) activating the infrared camera to acquire second infrared imaging data while the field of view of the infrared camera encompasses at least the second inspection area;

(k) activating a projector to project a pattern of light onto the second inspection area while the camera assembly is at the second location;

(l) activating the photogrammetry cameras to acquire second photogrammetry data while the field of view of the photogrammetry cameras encompasses at least a portion of the projected pattern of light on the second inspection area;

(m) stitching the first and second photogrammetry data together based on at least the first and second coordinate locations of the field of view of the photogrammetry cameras in the coordinate system of the composite structure;

(n) correlating the first and second infrared imaging data to the first and second photogrammetry data respectively; and

(o) stitching the first and second infrared imaging data together based on at least the results of steps (m) and (n).

8. The method as recited in claim 7, wherein said optical metrology comprises laser tracking.

9. The method as recited in claim 7, wherein said optical metrology comprises photogrammetry.

10. The method as recited in claim 8, wherein steps (c) and (i) collectively comprise:

placing optical targets on the composite structure;

directing respective pulses of light from a laser tracker toward the optical targets on the composite structure;

processing light returned from the optical targets on the composite structure to the laser tracker to determine first location data representing a coordinate location of the composite structure in a coordinate system of the laser tracker;

placing optical targets on the camera assembly;

directing respective pulses of light from the laser tracker toward the optical targets on the camera assembly;

processing light returned from the optical targets on the camera assembly to the laser tracker to determine second location data representing a coordinate location of the field of view of the photogrammetry cameras in the coordinate system of the laser tracker; and

computing the first and second coordinate locations of the field of view of the photogrammetry cameras in the coordinate system of the composite structure based on at least the first and second location data.

11. The method as recited in claim 10, wherein computing the first coordinate location of the field of view of the photogrammetry cameras in the coordinate system of the composite structure comprises identifying portions of the first photogrammetry data which correspond to light returned from optical targets on the composite structure.

12. A method for infrared thermographic inspection comprising:

(a) placing an infrared camera at a location where a field of view of the infrared camera encompasses an inspection area of a surface of a composite structure;

(b) locating the field of view of the infrared camera in a coordinate system of the composite structure using optical metrology;

(c) activating at least one flash lamp to output light that illuminates at least portions of the inspection area;

(d) after step (c), activating the infrared camera to acquire infrared imaging data while the field of view of the infrared camera encompasses at least the inspection area;

(e) correlating a set of the acquired infrared imaging data with three-dimensional model data representing characteristics of composite structure under the inspection area, said correlating being based on at least the location of the field of view of the infrared camera in the coordinate system of the composite structure;

(f) acquiring structural data from the three-dimensional model data, said structural data representing characteristics of the composite structure under the inspection area;

(g) correlating the acquired structural data with the set of acquired infrared imaging data;

(h) computing values of a parameter indicative of one of said characteristics of the composite structure under the inspection area based on at least the correlated structural data and infrared imaging data; and

(i) outputting parameter data representing the computed parameter values.

13. The method as recited in claim 12, further comprising identifying parameter data that is indicative of a defect.

14. The method as recited in claim 13, wherein the defect is delamination.

15. The method as recited in claim 13, wherein the defect is a porosity that equals or exceeds a specified threshold.

16. The method as recited in claim 12, wherein said structural data is thickness data and said parameter is porosity.

17. The method as recited in claim 16, wherein step (h) comprises:

selecting a data set from a thermal signature database, said data set representing a multiplicity of thermal signatures of a composite structure having said thickness and different respective porosity percentages;

identifying a thermal signature of the multiplicity that best matches a subset of the set of acquired infrared imaging data corresponding to said coordinate position;

retrieving from the thermal signature database porosity data associated with said identified thermal signature; and

associating the retrieved porosity data with said coordinate position.

18. The method as recited in claim 12, wherein the composite structure has a shape of a full barrel, further comprising rotating the composite structure about a longitudinal axis through a specified angle, ceasing rotation of the composite structure, and repeating steps (b) through (i) while the composite structure is stationary.

19. The method as recited in claim 12, wherein said optical metrology comprises laser tracking.

20. The method as recited in claim 12, wherein said optical metrology comprises photogrammetry.

21. A system for infrared thermographic inspection of a workpiece, comprising:

a robot comprising a movable robot base and an extendible robotic arm having a proximal end coupled to said robot base;

a frame mounted to a distal end of said robotic arm;

a target projector mounted to said frame;

a flash lamp mounted to said frame; and

a camera assembly mounted to said frame, said camera assembly comprising first and second camera pairs, said first camera pair comprising a first infrared camera and a first photogrammetry camera, and said second camera pair comprising a second infrared camera and a second photogrammetry camera,

wherein said target projector and said flash lamp are disposed between said first and second camera pairs.

22. The system as recited in claim 21, further comprising a laser tracker, a first set of at least three optical targets attached to said first camera pair and a second set of at least three optical targets attached to said second camera pair.

23. The system as recited in claim 22, further comprising a computer system configured to stitch first and second infrared imaging data acquired by said infrared camera based on at least location data acquired by said laser tracker and photogrammetry data acquired by said first and second photogrammetry cameras.