Aircraft inspection system with visualization and recording

• US 10,107,767 B1
• Filed: 06/14/2017
• Issued: 10/23/2018
• Est. Priority Date: 06/14/2017
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Laughlin, Brian D., Moore, Richard T., Franzman, Steven Curtis, Aguilar, David R., Juhnke, Elizabeth
• Primary Examiner(s): Chen, Frank
• Application Number:
  US15/622,756
• Time in Patent Office: 496 Days
• Field of Search: None
• CPC Class Codes: G01B 15/00 Measuring arrangements characterised by the use of electromagnetic waves or particle radiation, e.g. by the use of microwaves, X-rays, gamma rays or electrons characterised by the use of optical techniques G01B9/00, G01B11/00
• CPC Class Codes: G01M 5/0033 by determining damage, crack or wear
• CPC Class Codes: G01M 5/0075 by means of external apparatus, e.g. test benches or portable test systems G01M5/005 takes precedence
• CPC Class Codes: G01N 23/04 and forming images of the material
• CPC Class Codes: G01N 23/18 Investigating the presence of flaws defects or foreign matter
• CPC Class Codes: G01N 29/22 Details , e.g. general constructional or apparatus details
• CPC Class Codes: G01N 29/225 Supports, positioning or alignment in moving situation
• CPC Class Codes: G01N 29/4409 by comparison
• CPC Class Codes: G06Q 10/06395 Quality analysis or management
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance

Abstract

A method, system, and apparatus for a vehicle inspection system comprising a mobile inspection device, a display system, a graphical user interface configured to be displayed on the display system, and a controller. The controller is configured to identify a position of the mobile inspection device within a vehicle and receives a user input comprising a group of gestures made by a human operator with respect to an item in a group of items in a field of view of the mobile inspection device based on the position of the mobile inspection device. The controller creates a note at a location with respect to the vehicle in association with the item in the field of view of the mobile inspection device in which the note is assigned to the location with respect to the vehicle and displays the note on the graphical user interface for the mobile inspection device.

Claims

1. A vehicle inspection system comprising:

a mobile inspection device;

a display system for the mobile inspection device;

a graphical user interface configured to be displayed on the display system; and

a controller configured to operate with the mobile inspection device, wherein the controller is configured to identify a position of the mobile inspection device within a vehicle;

receive a user input comprising a group of gestures made by a human operator with respect to an item in a group of items in a field of view of the mobile inspection device based on the position of the mobile inspection device;

create a note at a location with respect to the vehicle in association with the item in the field of view of the mobile inspection device in which the note is assigned to the location with respect to the vehicle; and

display the note in association with the item in the field of view on the graphical user interface on the display system for the mobile inspection device.

2. The vehicle inspection system of claim 1 further comprising:

a storage system in which a history of notes is stored.

3. The vehicle inspection system of claim 1, wherein the controller is configured to identify a set of notes in the field of view of the mobile inspection device for a current position of the mobile inspection device and display the set of notes on the graphical user interface on the display system.

4. The vehicle inspection system of claim 3, wherein the set of notes identified is based on a policy applied to a role of the human operator of the mobile inspection device.

5. The vehicle inspection system of claim 1, wherein the controller is located in at least one of the mobile inspection device or a remote computer in communication with the mobile inspection device.

6. The vehicle inspection system of claim 1, wherein the display of the note in association with the item in the field of view on the graphical user interface on the display system for the mobile inspection device forms at least one of a mixed reality environment or an augmented reality environment.

7. The vehicle inspection system of claim 1, wherein the note comprises a graphical indicator and details selected from at least one of text, voice, an image, or a video.

8. The vehicle inspection system of claim 7, wherein the graphical indicator indicates a status selected from a group consisting of new, in progress, and completed.

9. The vehicle inspection system of claim 1, wherein the controller stores notes for the vehicle in a history, enabling root cause analysis to reduce future inconsistencies.

10. The vehicle inspection system of claim 1, wherein the controller displays notes for a group of locations over time for a group of vehicles, enabling viewing a trend of one or more inconsistencies in the group of locations.

11. The vehicle inspection system of claim 1, wherein the controller displays a group of metrics for the group of items.

12. The vehicle inspection system of claim 1, wherein the controller facilitates communication with a group of remote experts.

13. The vehicle inspection system of claim 1, wherein the mobile inspection device is selected from a group consisting of a mobile phone, a head mounted display, mixed reality smart glasses, and a tablet computer.

14. The vehicle inspection system of claim 1, wherein the user input is generated using at least one of a gesture recognition system, wired gloves, a stereo camera, a gesture based controller, a mouse, a track ball, a keyboard, a track pad, a touch screen tablet, a motion sensing input device, a cyber glove, a camera, or a microphone.

15. The vehicle inspection system of claim 1, wherein the note is generated during one of a final inspection prior to delivery to a customer, maintenance, an assembly of the vehicle, and a certification of the vehicle.

16. An aircraft inspection system comprising:

a mobile inspection device;

a user interface for the mobile inspection device; and

a controller configured to operate with the mobile inspection device, wherein the controller is configured to identify a position of the mobile inspection device within an aircraft;

receive a user input comprising a group of gestures made by a human operator with respect to an item in a field of view of the mobile inspection device based on the position of the mobile inspection device within the aircraft;

create a note in association with the item in a group of items in the field of view of the mobile inspection device; and

display the note in association with the item in the field of view on a display system for the mobile inspection device, enabling immersion in an augmented reality environment.

17. The aircraft inspection system of claim 16 further comprising:

a storage system in which a history of notes is stored.

18. The aircraft inspection system of claim 16, wherein the controller is configured to identify a set of notes in the field of view of the mobile inspection device for a current position of the mobile inspection device based on a policy applied to a role of the human operator of the mobile inspection device and display the set of notes on a graphical user interface on the display system.

19. The aircraft inspection system of claim 16, wherein the note comprises a graphical indicator and at least one of text, voice, an image, or a video.

20. The aircraft inspection system of claim 16, wherein the controller facilitates communication with a group of remote experts.

21. The aircraft inspection system of claim 16, wherein the note is generated during one of a final inspection prior to delivery to a customer, maintenance, an assembly of a vehicle, and a certification of the vehicle.

22. A method for inspecting a vehicle, the method comprising:

identifying a position of a mobile inspection device within the vehicle;

receiving a user input comprising a group of gestures made by a human operator with respect to an item in a group of items in a field of view of the mobile inspection device based on the position of the mobile inspection device;

creating a note at a location with respect to the vehicle in association with the item in the group of items in the field of view of the mobile inspection device in which the note is assigned to the location; and

displaying the note in association with the item in the field of view on a graphical user interface on a display system for the mobile inspection device.

23. The method of claim 22 further comprising:

storing the note in a storage system in which a history of notes is stored.

24. The method of claim 22 further comprising:

identifying a set of notes in the field of view of the mobile inspection device for a current position of the mobile inspection device; and

displaying the set of notes on the graphical user interface on the display system.

25. The method of claim 22, wherein a set of notes identified is based on a policy applied to a role of the human operator of the mobile inspection device.

26. The method of claim 22, wherein the displaying of the note in association with the item on the graphical user interface on the display system for the mobile inspection device forms at least one of a mixed reality environment or an augmented reality environment.

27. The method of claim 22, wherein the note comprises a graphical indicator and at least one of text, voice, an image, or a video;

and the graphical indicator indicates a status selected from a group consisting of new, in progress, and completed.