System and methods of inspecting an object

• US 9,607,370 B2
• Filed: 01/15/2014
• Issued: 03/28/2017
• Est. Priority Date: 01/15/2014
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Freeman, Philip L.
• Primary Examiner(s): Harvey, David
• Application Number:
  US14/156,019
• Publication Number:
  US 20150199802A1
• Time in Patent Office: 1,168 Days
• Field of Search: 348/129
• US Class Current: 1/1
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A system for use in inspecting an object is provided. The system includes at least one array of visual imaging devices configured to capture a plurality of two-dimensional images of the object. The array is configured to capture a first set of two-dimensional images over a first predetermined interval and a second set of two-dimensional images over a second predetermined interval that is after the first predetermined interval. The system also includes a computing device coupled to the at least one array of visual imaging devices. The computing device is configured to extract point clouds of the object from the first and second sets of two-dimensional images, generate a three-dimensional model of the object from the extracted point clouds, determine variations in the extracted point clouds from the first and second sets of two-dimensional images, and utilize the determined variations to detect potential anomalies in the three-dimensional model.

Claims

1. A system for use in inspecting an object, said system comprising:

at least one array of visual imaging devices configured to capture a plurality of two-dimensional images of the object, wherein the object is configured to translate relative to said at least one array, said at least one array configured to capture a first set of two-dimensional images over a first predetermined interval and a second set of two-dimensional images over a second predetermined interval that is after the first predetermined interval; and

a computing device coupled to said at least one array of visual imaging devices, said computing device comprising a processor and a memory having computer-executable instructions stored thereon, wherein, when executed by said processor, the computer-executable instructions cause said processor to;

determine moving portions and non-moving portions in the plurality of two-dimensional images;

extract point clouds of the object from the first and second sets of two-dimensional images, wherein the point clouds include the moving portions in the plurality of two-dimensional images;

generate a three-dimensional model of the object from the extracted point clouds;

determine variations in the extracted point clouds from the first and second sets of two-dimensional images; and

utilize the determined variations to detect potential anomalies in the three-dimensional model.

2. The system in accordance with claim 1, wherein the computer-executable instructions further cause said processor to:

define regions of interest on the three-dimensional model; and

extract portions of the point clouds of the object from the first and second sets of two-dimensional that correspond to the regions of interest.

3. The system in accordance with claim 2, wherein the computer-executable instructions further cause said processor to determine when variations between the portions of the point clouds in the regions of interest are greater than a predetermined threshold.

4. The system in accordance with claim 1, wherein the computer-executable instructions further cause said processor to perform a fine registration of the point clouds of the object from the first and second sets of two-dimensional images.

5. The system in accordance with claim 1, wherein the computer-executable instructions further cause said processor to perform change detection of the point clouds of the object from the first and second sets of two-dimensional images.

6. The system in accordance with claim 1, wherein the computer-executable instructions further cause said processor to designate regions of the object for further inspection when a potential anomaly is detected in the three-dimensional model.

7. The system in accordance with claim 1, wherein said at least one array of visual imaging devices comprises a first array at a first location and a second array at a second location, wherein the object is selectively moveable between the first and second locations.

8. A method of inspecting an object, said method comprising:

capturing, with at least one array of visual imaging devices, a first set of two-dimensional images of the object over a first predetermined interval, wherein the object is configured to translate relative to the at least one array;

capturing, with the at least one array of visual imaging devices, a second set of two-dimensional images of the object over a second predetermined interval that is after the first predetermined interval;

determining moving portions and non-moving portions in the plurality of two-dimensional images;

extracting point clouds of the object from the first and second sets of two-dimensional images, wherein the point clouds include the moving portions in the plurality of two-dimensional images;

generating a three-dimensional model of the object from the extracted point clouds;

determining variations in the extracted point clouds from the first and second sets of two-dimensional images; and

utilizing the determined variations to detect potential anomalies in the three-dimensional model.

9. The method in accordance with claim 8 further comprising capturing at least one image of the first set of two-dimensional images at a different orientation than images in the second set of two-dimensional images.

10. The method in accordance with claim 8, wherein extracting point clouds of the object comprises:

defining regions of interest on the three-dimensional model; and

extracting portions of the point clouds of the object from the first and second sets of two-dimensional that correspond to the regions of interest.

11. The method in accordance with claim 10, wherein determining variations in the point clouds comprises determining when variations between the portions of the point clouds in the regions of interest are greater than a predetermined threshold.

12. The method in accordance with claim 8, wherein generating a three-dimensional model of the object comprises performing a fine registration of the point clouds of the object from the first and second sets of two-dimensional images.

13. The method in accordance with claim 8, wherein determining variations in the extracted point clouds comprises performing change detection of the point clouds of the object from the first and second sets of two-dimensional images.

14. The method in accordance with claim 8, wherein utilizing the determined variations comprises designating regions of the object for further inspection when a potential anomaly is detected in the three-dimensional model.

15. A non-transitory computer-readable storage device having computer-executable instructions embodied thereon for use in inspecting an object, wherein, when executed by a computing device, the computer-executable instructions cause the computing device to:

determine moving portions and non-moving portions in a plurality of two-dimensional images of the object, wherein the plurality of two-dimensional images include a first set of two-dimensional images and a second set of two-dimensional images taken over separate predetermined intervals;

extract point clouds of the object from the first set and the second set of two-dimensional images, wherein the point clouds include the moving portions in the plurality of two-dimensional images;

generate a three-dimensional model of the object from the extracted point clouds;

determine variations in the extracted point clouds from the first and second sets of two-dimensional images; and

utilize the determined variations to detect potential anomalies in the three-dimensional model.

16. The non-transitory computer readable medium in accordance with claim 15 further comprising computer-executable instructions that cause the computing device to:

define regions of interest on the three-dimensional model; and

extract portions of the point clouds of the object from the first and second sets of two-dimensional that correspond to the regions of interest.

17. The non-transitory computer readable medium in accordance with claim 16 further comprising computer-executable instructions that cause the computing device to determine when variations between the portions of the point clouds in the regions of interest are greater than a predetermined threshold.

18. The non-transitory computer readable medium in accordance with claim 16 further comprising computer-executable instructions that cause the computing device to perform a fine registration of the point clouds of the object from the first and second sets of two-dimensional images.

19. The non-transitory computer readable medium in accordance with claim 16 further comprising computer-executable instructions that cause the computing device to perform change detection of the point clouds of the object from the first and second sets of two-dimensional images.

20. The non-transitory computer readable medium in accordance with claim 16 further comprising computer-executable instructions that cause the computing device to designate regions of the object for further inspection when a potential anomaly is detected in the three-dimensional model.