Automated paint quality control for aircraft

• US 10,643,329 B2
• Filed: 04/09/2018
• Issued: 05/05/2020
• Est. Priority Date: 04/09/2018
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Afrasiabi, Amir, Nguyen, Hieu Tat, Zangeneh-Khamooshi, Shabnam, Mani, Ashutosh
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US15/948,386
• Publication Number:
  US 20190311469A1
• Time in Patent Office: 757 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/30156 Vehicle coating
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

According to various embodiments, a technique for automated aircraft paint application quality inspection is presented. The technique includes retrieving from electronic persistent storage a control image depicting at least a portion of aircraft paint application for the aircraft under quality inspection; capturing a process image depicting at least a portion of aircraft paint application on the aircraft under quality inspection; detecting a plurality of features in the control image and the process image; registering the control image with the process image; detecting at least one difference between the control image and the process image; generating an output image, where the output image includes a depiction of the aircraft under quality inspection with the at least one difference annotated; and causing the output image to be displayed.

Claims

1. A method of automated aircraft paint application quality inspection, the method comprising:

retrieving from electronic persistent storage a control image depicting at least a portion of aircraft paint application for an aircraft under quality inspection;

placing at least one standardized marker on the aircraft under quality inspection;

capturing a process image depicting at least the portion of aircraft paint application on the aircraft under quality inspection;

capturing at least a first perspective image of at least the portion of the aircraft paint application under quality inspection from a first perspective and capturing at least a second perspective image of at least the portion of the aircraft paint application under quality inspection from a second perspective different from the first perspective;

detecting and removing at least one reflection from consideration as a difference between the control image and the process image by detecting different distances between the at least one standardized marker and candidate blobs in the first perspective image and the second perspective image;

detecting a plurality of features in the control image and the process image;

registering the control image with the process image;

detecting at least one difference between the control image and the process image;

generating an output image, wherein the output image comprises a depiction of the aircraft under quality inspection with the at least one difference annotated; and

causing the output image to be displayed.

2. The method of claim 1, further comprising remediating an anomaly corresponding to the at least one difference.

3. The method of claim 1, wherein the detecting a plurality of features comprises detecting a plurality of features using a Speeded Up Robust Features (SURF) algorithm.

4. The method of claim 1, wherein the registering the control image with the process image is performed using a RANdom SAmple Consensus (RANSAC) algorithm.

5. The method of claim 1, wherein the generating an output image comprises combining at least a portion of an image taken from a first perspective with at least a portion of an image taken from a second perspective different from the first perspective.

6. The method of claim 1, wherein the control image comprises a combination of a first control image taken from a first perspective and a second control image taken from a second perspective different from the first perspective, and wherein the process image comprises a combination of a first process image taken from a third perspective and a second process image taken from a fourth perspective different from the third perspective.

7. The method of claim 1, wherein the detecting at least one difference comprises performing a pixel-by-pixel comparison between the control image and the process image.

15. The method of claim 1, wherein the at least one standardized marker comprises an ArUco marker.

16. The method of claim 1, wherein the causing the output image to be displayed comprises displaying the output image on a virtual reality headset.

17. The method of claim 1, wherein the candidate blobs are limited to blobs that are roughly conic section in shape.

8. A system for automated aircraft paint application quality inspection, the system comprising:

electronic persistent storage storing a control image depicting at least a portion of aircraft paint application for the aircraft under quality inspection;

a plurality of cameras disposed to capture multiple images of the aircraft under quality inspection; and

at least one electronic processor, communicatively coupled to the electronic persistent memory and the plurality of cameras, that executes instructions to perform operations comprising;

retrieving from the electronic persistent storage the control image;

placing at least one standardized marker on the aircraft under quality inspection;

capturing, by the plurality of cameras, a process image depicting at least a portion of aircraft paint application on the aircraft under quality inspection;

capturing at least a first perspective image of at least the portion of the aircraft paint application under quality inspection from a first perspective and capturing at least a second perspective image of at least the portion of the aircraft paint application under quality inspection from a second perspective different from the first perspective;

detecting and removing at least one reflection from consideration as a difference between the control image and the process image by detecting different distances between the at least one standardized marker and candidate blobs in the first perspective image and the second perspective image;

detecting a plurality of features in the control image and the process image;

registering the control image with the process image;

detecting at least one difference between the control image and the process image;

generating an output image, wherein the output image comprises a depiction of the aircraft under quality inspection with the at least one difference annotated; and

causing the output image to be displayed.

9. The system of claim 8, wherein the operations further comprise directing a user to remediate an anomaly corresponding to the at least one difference.

10. The system of claim 8, wherein the detecting a plurality of features comprises detecting a plurality of features using a Speeded Up Robust Features (SURF) algorithm.

11. The system of claim 8, wherein the registering the control image with the process image is performed using a RANdom SAmple Consensus (RANSAC) algorithm.

12. The system of claim 8, wherein the generating an output image comprises combining at least a portion of an image taken from a first perspective with at least a portion of an image taken from a second perspective different from the first perspective.

13. The system of claim 8, wherein the control image comprises a combination of a first control image taken from a first perspective and a second control image taken from a second perspective different from the first perspective, and wherein the process image comprises a combination of a first process image taken from a third perspective and a second process image taken from a fourth perspective different from the third perspective.

14. The system of claim 8, wherein the detecting at least one difference comprises performing a pixel-by-pixel comparison between the control image and the process image.

18. The system of claim 8, wherein the at least one standardized marker comprises an ArUco marker.

19. The system of claim 8, wherein the causing the output image to be displayed comprises displaying the output image on a virtual reality headset.

20. The system of claim 8, wherein the candidate blobs are limited to blobs that are roughly conic section in shape.