System and method for generating enhanced stereograhic videos of aircraft build processes

• US 10,621,715 B2
• Filed: 10/11/2019
• Issued: 04/14/2020
• Est. Priority Date: 07/13/2016
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Glatfelter, John W., Choi, Chong
• Primary Examiner(s): Perlman, David
• Application Number:
  US16/600,271
• Publication Number:
  US 20200043158A1
• Time in Patent Office: 186 Days
• Field of Search: None
• CPC Class Codes: G05B 19/41805 characterised by assembly
• CPC Class Codes: G05B 19/4183 characterised by data acquisition, e.g. workpiece identification
• CPC Class Codes: G05B 19/41885 characterised by modeling, simulation of the manufacturing system
• CPC Class Codes: G05B 2219/31472 Graphical display of process
• CPC Class Codes: G05B 23/0235 based on a comparison with predetermined threshold or range, e.g. "classical methods", carried out during normal operation; threshold adaptation or choice; when or how to compare with the threshold
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

Provided is a system and method for generating enhanced stereographic videos of aircraft build processes. Specifically, the system comprises a stereoscopic recording device configured to capture a plurality of stages of an aircraft build process. The system further comprises one or more processors, memory, and one or more programs stored in the memory that comprise instructions for execution by the system to build a stereographic library including repositories of 3D video corresponding to the plurality of stages of the aircraft build process. The system then generates an enhanced walkthrough video of the aircraft build process. The enhanced walkthrough video may include a parallax grid overlay and/or a thermal scan overlay integrated into the video. The system may then analyze the enhanced walkthrough video using post-processing analytics to identify anomalies and irregularities that occurred during the aircraft build process.

Claims

1. A system, comprising:

one or more processors;

memory; and

one or more programs stored in the memory, the one or more programs comprising instructions for;

building a stereographic library including repositories of 3D video, the repositories of 3D video corresponding to a plurality of stages of an aircraft build process;

generating an enhanced walkthrough video of the aircraft build process, the enhanced walkthrough video including a parallax grid overlay integrated into the video, the parallax grid overlay including a plurality of parallax lines, wherein each set of parallax lines can be activated or deactivated as needed; and

analyzing the enhanced walkthrough video using post-processing analytics.

2. The system of claim 1, wherein the post-processing analytics includes analyzing patterns and shapes to detect foreign object damage.

3. The system of claim 1, wherein the post-processing analytics includes analyzing patterns and shapes to determine assembly and sub-assembly compliance.

4. The system of claim 1, wherein the post-processing analytics includes analyzing patterns and shapes to determine thermal gradient compliance.

5. The system of claim 4, wherein determining thermal gradient compliance includes matching thermal heat to a thermal heat database storing therein expected temperature gradient information, for example stored within a server and/or database.

6. The system of claim 1, wherein the system is further configured to provide remote in-process initial inspections capabilities.

7. The system of claim 1, wherein the parallax grid overlay is configured such that accurate real-life measurements for locations, spacing, and aircraft structures can be extracted from the enhanced walkthrough video.

8. The system of claim 1, wherein the post-processing analytics includes scanning an image for standard structures by matching the shapes in the image with shape information stored in a standard shape database in order to confirm correct proper installation of structures.

9. The system of claim 8, wherein the standard shape database has stored therein expected shapes, for example stored within a server and/or database.

10. The system of claim 9, wherein the expected shapes comprise expected shapes for any one or more of:

assemblies, sub-assemblies, and customer unique options, and wherein optionally the expected shapes are derived from a computer-aided design and/or computer-aided manufacturing system which maintains the “

as-built”

configuration of an aircraft.

11. A method comprising:

building a stereographic library including repositories of 3D video, the repositories of 3D video corresponding to a plurality of stages of an aircraft build process,

generating an enhanced walkthrough video of the aircraft build process, the enhanced walkthrough video including a parallax grid overlay integrated into the video, the parallax grid overlay including a plurality of parallax lines, wherein each set of parallax lines can be activated or deactivated as needed; and

analyzing the enhanced walkthrough video using post-processing analytics.

12. The method of claim 11, wherein the post-processing analytics includes analyzing patterns and shapes to detect foreign object damage.

13. The method of claim 11, wherein the post-processing analytics includes analyzing patterns and shapes to determine assembly and sub-assembly compliance.

14. The method of claim 11, wherein the post-processing analytics includes analyzing patterns and shapes to determine thermal gradient compliance.

15. The method of claim 14, wherein determining thermal gradient compliance includes matching thermal heat to a thermal heat database storing therein expected temperature gradient information, for example stored within a server and/or database.

16. The method of claim 11, wherein the enhanced walkthrough video is configured to allow remote in-process initial inspections.

17. The method of claim 11, wherein the parallax grid is configured such that accurate real-life measurements for locations, spacing, and aircraft structures can be extracted from the enhanced walkthrough video.

18. The method of claim 11, wherein the post-processing analytics includes scanning an image for standard structures by matching the shapes in the image with shape information stored in a standard shape database in order to confirm correct proper installation of structures.

19. The method of claim 11, wherein the standard shape database has stored therein expected shapes, for example stored within a server and/or database.

20. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:

building a stereographic library including repositories of 3D video, the repositories of 3D video corresponding to a plurality of stages of an aircraft build process;

generating an enhanced walkthrough video of the aircraft build process, the enhanced walkthrough video including a parallax grid overlay integrated into the video, the parallax grid overlay including a plurality of parallax lines, wherein each set of parallax lines can be activated or deactivated as needed; and

analyzing the enhanced walkthrough video using post-processing analytics.