In-process fault inspection using augmented reality

• US 9,916,650 B2
• Filed: 10/14/2015
• Issued: 03/13/2018
• Est. Priority Date: 10/14/2014
• Status: Active Grant

• Current Assignee: Airbus Defence and Space GmbH
• Original Assignee: Airbus Defence and Space GmbH
• Inventors: Engel, Franz
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US14/883,001
• Publication Number:
  US 20160104276A1
• Time in Patent Office: 881 Days
• Field of Search: 382141, 382145, 382147, 382149, 382152, 382209, 382278, 348 86, 348125
• CPC Class Codes: G01N 2021/8893 providing a video image and a processed signal for helping visual decision
• CPC Class Codes: G01N 21/8851 Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002
• CPC Class Codes: G01N 21/9515 Objects of complex shape, e.g. examined with use of a surface follower device measuring contours and curvatures G01B11/24
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0004 Industrial image inspection

Abstract

An arrangement for visual fault inspection of at least one component includes a fault identification unit for identifying a structural fault of the component and for determining at least one piece of fault information, and an overlay device connected to the fault identification unit, configured for context-dependent overlay of the fault information in a component image in real-time.

Claims

1. An arrangement for visual fault inspection of at least one component, comprising:

a fault identification unit configured to identify a structural fault in the component and to determine at least one item of fault information;

an overlay device connected to the fault identification unit and configured to overlay the at least one item of fault information in a context-dependent manner in a component image in real-time,

a positioning-finding unit configured to determine exact positioning information about a position of the component and about a position of the user;

a processor unit connected to the fault identification unit and to the position-finding unit, the processor unit being configured to process the at least one item of fault information and the positioning information and to integrate the processed information in the component image; and

a visualisation unit connected to the processor unit configured to display the component image containing the integrated information of the processor unit.

2. The arrangement according to claim 1, further comprising:

a capture unit connected to the processor unit and configured to capture the component image.

3. The arrangement according to claim 1, wherein the component image corresponds to a real-time image of the component.

4. The arrangement according to claim 1, wherein the visualisation unit comprises one of a portable computer, eyeglasses, contact lenses, a portable screen, and a portable transparent display.

5. The arrangement according to claim 1, wherein the at least one item of fault information comprises pinpointing or coordinates of the structural fault.

6. The arrangement according to claim 1, wherein the at least one item of fault information comprises the severity of the structural fault.

7. The arrangement according to claim 1, wherein the positioning-finding unit comprises reference markers for calculating the position of the component and of the user.

8. The arrangement according to claim 1, wherein the processor unit comprises a synchronising unit configured to synchronise the movement of the visualisation unit and/or the direction of view of the visualisation unit with the displayed component image.

9. The arrangement according to claim 1, wherein the visualisation unit is configured for pinpointing the structural fault on the displayed component image.

10. A method for visual fault inspection of a component, the method comprising:

identifying a structural fault on the component;

determining at least one item of fault information of the structural fault using a fault identification unit;

overlaying the at least one item of fault information in context-dependent manner in a component image in real-time;

determining exact information about a position of the component and a position of the user by a positioning-finding unit;

processing the fault information and the information about the position of the component and the position of the user;

integrating the processed information in a component image by a processor unit; and

displaying the component image containing the integrated information by a visualisation unit in real-time.

11. The method according to claim 10, wherein the component image is captured in real time.

12. The method according to claim 10, wherein the displayed component image moves according to the movement of the visualisation unit and/or the direction of view of the visualisation unit.

13. The method according to claim 10, wherein the processing of the fault information comprises:

categorising the structural fault;

determining whether countermeasures are necessary; and

initiating counter measures depending on the determination.