System and method for generating enhanced stereographic videos of aircraft build processes

• US 10,445,867 B2
• Filed: 07/13/2016
• Issued: 10/15/2019
• Est. Priority Date: 07/13/2016
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Glatfelter, John W., Choi, Chong
• Primary Examiner(s): Perlman, David
• Application Number:
  US15/209,733
• Publication Number:
  US 20180018764A1
• Time in Patent Office: 1,189 Days
• Field of Search: None
• CPC Class Codes: G05B 19/41805 characterised by assembly
• CPC Class Codes: G05B 19/4183 characterised by data acquisition, e.g. workpiece identification
• CPC Class Codes: G05B 19/41885 characterised by modeling, simulation of the manufacturing system
• CPC Class Codes: G05B 2219/31472 Graphical display of process
• CPC Class Codes: G05B 23/0235 based on a comparison with predetermined threshold or range, e.g. "classical methods", carried out during normal operation; threshold adaptation or choice; when or how to compare with the threshold
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

Provided is a system and method for generating enhanced stereographic videos of aircraft build processes. Specifically, the system comprises a stereoscopic recording device configured to capture a plurality of stages of an aircraft build process. The system further comprises one or more processors, memory, and one or more programs stored in the memory that comprise instructions for execution by the system to build a stereographic library including repositories of 3D video corresponding to the plurality of stages of the aircraft build process. The system then generates an enhanced walkthrough video of the aircraft build process. The enhanced walkthrough video may include a parallax grid overlay and/or a thermal scan overlay integrated into the video. The system may then analyze the enhanced walkthrough video using post-processing analytics to identify anomalies and irregularities that occurred during the aircraft build process.

Claims

1. A system, comprising:

a stereoscopic camera, the stereoscopic camera configured to capture a plurality of stages of an aircraft build process;

one or more processors;

memory; and

one or more programs stored in the memory, the one or more programs comprising instructions for;

building a stereographic library including repositories of 3D video organized by tail number, the repositories of 3D video corresponding to the plurality of stages of the aircraft build process;

generating an enhanced walkthrough video of the aircraft build process, the enhanced walkthrough video including a parallax grid overlay integrated into the video, wherein each set of parallax lines in the parallax grid is created and stored on a separate video layer such that each set of parallax lines can be activated or deactivated as needed; and

analyzing the enhanced walkthrough video using post-processing analytics to identify anomalies and irregularities that occurred during the aircraft build process.

2. The system of claim 1, wherein the post-processing analytics includes analyzing patterns and shapes to detect foreign object damage.

3. The system of claim 1, wherein the post-processing analytics includes analyzing patterns and shapes to determine assembly and sub-assembly compliance.

4. The system of claim 1, wherein the post-processing analytics includes analyzing patterns and shapes to determine thermal gradient compliance.

5. The system of claim 1, wherein the system is further configured to provide remote in-process initial inspections capabilities.

6. The system of claim 1, wherein the parallax grid overlay is configured such that accurate real-life measurements for locations, spacing, and aircraft structures can be extracted from the enhanced walkthrough video.

7. The system of claim 1, wherein the parallax grid overlay includes a plurality of parallax lines determined automatically using autofocus,

wherein the plurality of parallax lines is organized as one or more sets of parallax lines,

wherein each set of parallax lines is stored as a separate video layer in the parallax grid overlay.

21. The system of claim 1, wherein the post-processing analytics includes scanning an image for standard structures by matching the shapes in the image with shape information stored in a standard shape database in order to confirm correct proper installation of structures.

22. The system of claim 21, wherein the standard shape database has stored therein expected shapes, for example stored within a server and/or database.

23. The system of claim 22, wherein the expected shapes comprise expected shapes for any one or more of:

assemblies, sub-assemblies, and customer unique options, and wherein optionally the expected shapes are derived from a computer-aided design and/or computer-aided manufacturing system which maintains the “

as-built”

configuration of an aircraft.

24. The system of claim 1, wherein the post-processing analytics includes searching and comparing a thermal scan overlay with a thermal heat database to identify temperature gradients that are not in compliance within thresholds of expected temperatures.

25. The system of claim 24, wherein the thermal heat database has stored therein expected temperature gradient information, for example stored within a server and/or database.

26. The system of claim 25, wherein the expected temperature gradients comprise expected temperature gradients of various areas of an aircraft, optionally for each aircraft and build-stage.

8. A method comprising:

capturing a plurality of stages of an aircraft build process via a stereoscopic camera;

building a stereographic library including repositories of 3D video organized by tail number, the repositories of 3D video corresponding to the plurality of stages of the aircraft build process;

generating an enhanced walkthrough video of the aircraft build process, the enhanced walkthrough video including a parallax grid overlay integrated into the video, wherein each set of parallax lines in the parallax grid is created and stored on a separate video layer such that each set of parallax lines can be activated or deactivated as needed; and

analyzing the enhanced walkthrough video using post-processing analytics to identify anomalies and irregularities that occurred during the aircraft build process.

9. The method of claim 8, wherein the post-processing analytics includes analyzing patterns and shapes to detect foreign object damage.

10. The method of claim 8, wherein the post-processing analytics includes analyzing patterns and shapes to determine assembly and sub-assembly compliance.

11. The method of claim 8, wherein the post-processing analytics includes analyzing patterns and shapes to determine thermal gradient compliance.

12. The method of claim 8, wherein the enhanced walkthrough video is configured to allow remote in-process initial inspections.

13. The method of claim 8, wherein the parallax grid is configured such that accurate real-life measurements for locations, spacing, and aircraft structures can be extracted from the enhanced walkthrough video.

14. The method of claim 8, wherein the parallax grid includes a plurality of parallax lines determined automatically using autofocus,

wherein the plurality of parallax lines is organized as one or more sets of parallax lines,

wherein each set of parallax lines is stored as a separate video layer in the parallax grid overlay.

15. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:

capturing a plurality of stages of an aircraft build process via a stereoscopic camera

building a stereographic library including repositories of 3D video organized by tail number, the repositories of 3D video corresponding to the plurality of stages of the aircraft build process;

generating an enhanced walkthrough video of the aircraft build process, the enhanced walkthrough video including a parallax grid overlay integrated into the video, wherein each set of parallax lines in the parallax grid is created and stored on a separate video layer such that each set of parallax lines can be activated or deactivated as needed; and

analyzing the enhanced walkthrough video using post-processing analytics to identify anomalies and irregularities that occurred during the aircraft build process.

16. The non-transitory computer readable storage medium of claim 15, wherein the post-processing analytics includes analyzing patterns and shapes to detect foreign object damage.

17. The non-transitory computer readable storage medium of claim 15, wherein the post-processing analytics includes analyzing patterns and shapes to determine assembly and sub-assembly compliance.

18. The non-transitory computer readable storage medium of claim 15, wherein the post-processing analytics includes analyzing patterns and shapes to determine thermal gradient compliance.

19. The non-transitory computer readable storage medium of claim 15, wherein the parallax grid is configured such that accurate real-life measurements for locations, spacing, and aircraft structures can be extracted from the enhanced walkthrough video.

20. The non-transitory computer readable storage medium of claim 15, wherein the parallax grid includes a plurality of parallax lines determined automatically using autofocus,

wherein the plurality of parallax lines is organized as one or more sets of parallax lines,

wherein each set of parallax lines is stored as a separate video layer in the parallax grid overlay.