Assembly inspection system and method

• US 9,187,188 B2
• Filed: 12/16/2013
• Issued: 11/17/2015
• Est. Priority Date: 07/02/2013
• Status: Active Grant

• Current Assignee: Premium Aerotec GmbH (Airbus SE)
• Original Assignee: Premium Aerotec GmbH (Airbus SE)
• Inventors: Berndt, Dirk, Richter, Ingo, Schomacker, Michael, Schwanzar, Sebastian, Garmann, Daniel, Sauer, Steffen, Trostmann, Erik
• Primary Examiner(s): Cheung, Mary
• Assistant Examiner(s): Trivedi, Atul
• Application Number:
  US14/107,435
• Publication Number:
  US 20150012171A1
• Time in Patent Office: 701 Days
• Field of Search: 701/3, 701 13- 14, 701/32.9, 701/36, 701/39
• US Class Current: 1/1
• CPC Class Codes: B64F 5/60 Testing or inspecting aircraft components or systems
• CPC Class Codes: G05B 2219/37205 Compare measured, vision data with computer model, cad data
• CPC Class Codes: G05B 2219/45071 Aircraft, airplane, ship cleaning manipulator, paint stripping
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/001 using an image reference approach

Abstract

A method for inspecting assembly of components in a structure includes acquiring a visual representation of at least a portion of the structure, and saving an electronic file of the visual representation on a computer readable medium. A three-dimensional design of the structure, which contains information on a proper position of the components within the structure, is accessed. The visual representation is compared with the three-dimensional design using a computer, and a feedback indicating a result of the comparison is generated.

Claims

1. A method for inspecting assembly of components in an aircraft structure, comprising:

acquiring a visual representation of at least a portion of the structure that includes a plurality of components therein;

saving an electronic file of the visual representation on a computer readable medium;

accessing a three-dimensional design of the structure, the three dimensional design containing information on a proper position of each of the plurality of components within the structure;

comparing the visual representation with the three-dimensional design using a computer by graphically superimposing an image related to the visual representation with a second image related to the three-dimensional design to determine whether each of the plurality of components encompassed by the visual representation is in a proper position in the structure, as determined by a position of each corresponding component encompassed in the three-dimensional design; and

generating a feedback indicating a result of the comparison.

2. The method of claim 1, further comprising automatically compiling a list of inspection stations relative to the structure based on the three-dimensional design.

3. The method of claim 2, further comprising compiling a list of inspection stations that are inaccessible for acquisition of the visual representation.

4. The method of claim 2, wherein the acquiring of the visual representation is performed using an optical sensor.

5. The method of claim 4, wherein the optical sensor is a stereoscopic sensor having at least two digital cameras.

6. The method of claim 4, wherein the method further comprises developing an inspection path for a conveyance device configured to selectively position the visual sensor at each inspection station in response to commands provided by the controller.

7. The method of claim 1, wherein comparing the visual representation with the three-dimensional design includes superimposing line-view images of the structure with line-view images rendered from the three-dimensional design, and comparing the superimposed images for agreement of edges appearing therein.

8. A method for automatic inspection of a real aircraft structure against a computer aided design (CAD) model of the real aircraft structure, comprising:

retrieving CAD data of the model;

generating a checking plan that includes a plurality of inspection stations;

generating a robot plan for a robot carrying a sensor array disposed to acquire visual information from the real aircraft structure, the robot plan including a plurality of positions, each position corresponding to a particular inspection station;

providing the robot plan to a robot controller;

providing the checking plan to a data processing unit;

executing the robot plan to obtain visual information from the real aircraft structure;

comparing the visual information with the CAD data at each inspection station, wherein comparing the visual information with the CAD date includes graphically superimposing the visual information with an image related to the CAD data to determine whether each of a plurality of components that make up the real aircraft structure is in a proper position with respect to corresponding components encompassed in the CAD data of the model; and

generating an output indicative of component deviations when the comparison of the visual information with the CAD data indicates that a component of the real aircraft structure has at least one of an incorrect position, incorrect shape, or is not present on the real aircraft structure.

9. The method of claim 8, wherein obtaining visual information from the real aircraft structure includes acquiring an electronic image of a portion of the real aircraft structure using a camera that is part of the sensor array.

10. The method of claim 9, wherein the electronic image is a three-dimensional image obtained using a second camera disposed at a distance and at an angle relative to the camera such that the two cameras can together provide a three-dimensional perspective view of the portion of the real aircraft structure.

11. The method of claim 10, further comprising:

converting visual information provided by the sensor array at each inspection station visited by the sensor array into a line-view image of the real aircraft structure;

generating a line-view image of the design at the same distance and using the same perspective as a respective inspection station visited by the sensor array that provided the visual information corresponding to that particular inspection station;

superimposing the line-view image from the sensor array with the line-view image from the design of the real aircraft structure; and

determining that a fault in an assembly of the real aircraft structure is present when at least one edge in the line-view image of the real aircraft structure deviates from a corresponding edge in the line-view image of the design.

12. The method of claim 11, further comprising portraying a result of the determination that a fault in the assembly has occurred by representing desired and actual location and/or orientation of a component to which the at least one edge belongs in a single image that is provided for viewing by an inspector.

13. The method of claim 8, further comprising tracking at least one of a position, orientation, speed and trajectory of the sensor array relative to the real aircraft structure using an array of tracker targets associated with the sensor array and a positional sensor disposed to sense the array of tracker targets and provide positional information to a controller.

14. The method of claim 8, further comprising compiling a plurality of exceptions containing a subset of the inspection stations that are inaccessible to the sensor array.

15. The method of claim 8, wherein the obtaining of visual information from the real aircraft structure includes acquiring at least one three-dimensional image such that edges and a shape of assembled components in the real aircraft structure can be identified.

16. An inspection system adapted to inspect an aircraft structure, the aircraft structure having a plurality of components therein, the inspection system comprising:

a non-transitory, computer-readable database containing computer readable information therein indicative of a design of the aircraft structure;

an image capturing device configured to capture visual information from the aircraft structure;

a conveyance device configured to selectively carry and position the sensor array relative to the aircraft structure; and

a controller associated with the database, the sensor array, and the conveyance device, the controller disposed to compare the visual information to the information indicative of the design by graphically superimposing an image related to the visual information with a second image related to the three-dimensional design to determine whether each of the plurality of components encompassed by the visual information is in a proper position in the structure, as determined by a position of each corresponding component encompassed in the three-dimensional design.

17. The inspection system of claim 16, wherein the image capturing device includes a camera having a sensor array, and wherein the inspection system further comprises:

a structured light projector configured to provide a lit pattern onto the aircraft structure within a viewing angle of the camera; and

a data processing unit associated with the camera and structured light projector.

18. The inspection system of claim 17, further comprising a second camera disposed at a distance and at an angle relative to the camera such that the two cameras can together provide a three-dimensional perspective view of the objects.

19. The inspection system of claim 16, further comprising:

an array of tracker targets associated with the sensor array; and

a positional sensor disposed to sense the array of tracker targets such that the controller can determine a position, orientation, speed and/or trajectory of the sensor array with respect to the positional sensor based on information provided to the controller by the positional sensor.

20. The inspection system of claim 16, wherein the conveyance device is a robot having multiple axes of motion, wherein the sensor array is connected to a moveable arm of the robot, and wherein the controller is disposed to provide motion commands to the robot that are effective in moving the sensor array to a desired position and orientation relative to the aircraft structure.

21. The inspection system of claim 16, wherein the sensor array is configured to acquire:

three-dimensional images so that edges and a shape of assembled components in the aircraft structure can be identified by the controller; and

two-dimensional images so that a location of edges of the assembled components and openings in the assembled components can be identified by the controller.

22. The inspection system of claim 16, wherein the controller is further disposed to:

convert the visual information provided by the sensor array at each inspection station visited by the sensor array into a line-view image of the aircraft structure;

wherein generating inspection station information based on the design of the aircraft structure includes generating a line-view image of the design at the same distance and using the same perspective as a respective inspection station visited by the sensor array that provided the visual information corresponding to that particular inspection station;

wherein compare the visual information with the inspection station information includes superimposing the line-view image from the sensor array with the line-view image from the design of the aircraft structure; and

determine that a fault in an assembly of the aircraft structure is present when at least one edge in the line-view image of the aircraft structure deviates from a corresponding edge in the line-view image of the design.