Aircraft maintenance systems and methods using wearable device

• US 10,065,750 B2
• Filed: 02/10/2015
• Issued: 09/04/2018
• Est. Priority Date: 02/10/2015
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Kathirvel, Kannan, Mahalingaiah, Pradeep, Christensen, Michael
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Li, Ce Li
• Application Number:
  US14/618,181
• Publication Number:
  US 20160229562A1
• Time in Patent Office: 1,302 Days
• CPC Class Codes: B64F 5/60 Testing or inspecting aircraft components or systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/141 Setup of application sessions admission control or resource allocation in data switching networks H04L47/70
• CPC Class Codes: H04R 1/028 associated with devices performing functions other than acoustics, e.g. electric candles
• CPC Class Codes: H04R 2460/13 Hearing devices using bone conduction transducers

Abstract

Methods and systems are provided for managing maintenance activities of an aircraft using a wearable device. In one embodiment, a method comprises: receiving, at the wearable device, part data from at least one of a video recording device and an audio recording device of the wearable device; communicating by the wearable device with at least one of an onboard avionics system, an airline operations system, and an inventory management system based on the part data; and presenting maintenance information to a user via at least one of a display system, and an audio system of the wearable device based on the communicating.

Claims

1. A method for managing maintenance activities by a maintenance module on an aircraft coupled with a wearable device receiving both audio and video data, comprising:

configuring the maintenance module to receive both the audio and video data from one or more input devices for use with obtaining data of maintenance activities from a plurality of computing systems coupled to the maintenance module, the computing systems comprising;

onboard aircraft systems, airline operation systems, and inventory systems wherein the inventory systems store information about aircraft parts and tools used in maintenance activities;

generating, by a part data determination module within the maintenance module, part data from an interaction by use with the one or more input devices wherein the input devices comprise devices capable of receiving a voice interaction by use and a video interaction by use;

processing, by the one or more input devices, the voice and video interaction by the use to identify the part data wherein the voice interaction comprise;

voiced data of the part data by use and the video interaction comprise;

image data of the part data by use;

receiving, by the part data determination module at the wearable device, voice and image data of the part data from at least the one or more input devices which further comprise;

a video recording device and an audio recording device associated with the wearable device wherein the video recording device determines the part data by image processing solutions of at least the image data and the audio recording device determines a part number by speech processing solutions of at least the voice data;

confirming, by the part data determination module, at least a number of a replacement unit from the part data determined by either the image processing or speech processing solutions wherein the confirming is based on at least information which comprise;

aircraft information and maintenance information loaded into the maintenance module;

communicating, after the confirming by the part data determination module, by the wearable device via an information processing module coupled to the part data determination module, requests for part information associated with the part data to the computing systems, the information processing module further configured to perform the step of;

receiving part information related to the part data from the computing systems, processing the part information from the computing systems to generate user interface data to integrate, in a first instance the part information into a user interface for display and in a second instance the part information into audio data for audio notifications;

presenting maintenance information, by the information processing module, via the user interface for display by a display system, and audio notifications by an audio system of the wearable device;

configuring, by the information processing module from the maintenance information presented, a part option from a user selection of a set of part options via the one or more input devices, the set of part options comprising;

to order a new part, to request a maintenance procedure for a part, and to report a maintenance activity status to one or more of the computing systems; and

receiving part option data in response to the user selection of the part option wherein the information processing module communicates maintenance procedure data with the part option data to the computing systems.

2. The method of claim 1, further comprising identifying a part number from the part data, and wherein the communicating is based on the part number.

3. The method of claim 2, wherein the identifying is based on a speech recognition technique.

4. The method of claim 2, wherein the identifying is based on a video processing technique.

5. The method of claim 1, wherein the presenting maintenance information comprises displaying maintenance symbols in a field of view of the user via the display system.

6. The method of claim 1, wherein the presenting maintenance information comprises playing maintenance instructions to the user via the audio system.

7. The method of claim 6, wherein the audio system includes at least one of a bone conduction device of the wearable device and a conventional audio device of the wearable device.

8. The method of claim 1, further comprising establishing a connection between the wearable device and the onboard avionics system, and wherein the communicating is based on the connection.

9. The method of claim 1, further comprising establishing a connection between the wearable device the airline operations system, and wherein the communicating is based on the connection.

10. The method of claim 1, further comprising establishing a connection between the wearable device and the inventory management system, and wherein the communicating is based on the connection.

11. A system for managing maintenance of an aircraft, comprising:

a wearable device comprising a display device, an audio system, and at least one of an audio recording device and a video recording device, wherein the wearable device is configured to;

generate, via a maintenance module, part data from an interaction by use of the wearable device with one or more input devices coupled to the maintenance module wherein the input devices comprise devices capable of receiving inputs of a voice interaction by the use and a video interaction by the use;

process, via the maintenance module, the inputs from the one or more input devices of the voice and video interaction by the use to identify the part data wherein the voice interaction comprise;

voiced data of the part data by the use and the video interaction comprise;

image data of the part data by the use;

receive, via the maintenance module, voice and image data of the part data from at least the one or more input devices which further comprise;

the video recording device and the audio recording device wherein the video recording device determines the part data by image processing solutions of at least the image data and the audio recording device determines a part number by speech processing solutions of at least the voice data;

confirm, via the maintenance module, an option of a set of options of ordering at least a number of a replacement unit from the part data determined by either the image processing or speech processing solutions wherein the confirming is based on at least information which comprise;

aircraft information and maintenance information loaded into the maintenance module wherein the set of options further comprises;

to request a maintenance procedure for a part, and to report a maintenance activity status; and

communicate with at least one of an onboard avionics system, an airline operations system, and an inventory management system based on the part data, and present maintenance information to a user via at least one of the display system and a bone conduction device of the wearable device.

12. The system of claim 11, wearable device is configured to identify part number from the part data, and wherein communicate based on the part number.

13. The system of claim 12, wherein the wearable device is configured to identify the part number based on a speech recognition technique.

14. The system of claim 12, wherein the wearable device is configured to identify the part number based on a video processing technique.

15. The system of claim 11, wherein the wearable device is configured to present maintenance information by displaying maintenance symbols in a field of view of the user via the display system.

16. The system of claim 11, wherein the wearable device is configured to present maintenance information by playing maintenance instructions to the user via the audio system.

17. The system of claim 11, wherein the audio system is a bone conduction system.

18. The system of claim 11, wherein the wearable device is configured to establish a connection between the wearable device and the onboard avionics system, and communicate based on the connection.

19. The system of claim 11, wherein the wearable device is configured to establish a connection between the wearable device the airline operations system, and communicate based on the connection.

20. The system of claim 11, wherein the wearable device is configured to establish a connection between the wearable device and the inventory management system, and communicate based on the connection.

21. A system for managing maintenance of an aircraft, comprising:

a wearable device to communicate with one or more computing systems via input/output interfaces wherein the wearable device comprises;

a maintenance module, an audio device, a display device, and at least one of an audio recording device and a video recording device wherein the display device is configurable such that information pertaining to a maintenance activity is displayed in a field of view of an user irrespective of a viewing direction of the user wherein the computing systems comprise;

an onboard system that stores information about the aircraft, an airline operations system that stores departure and arrival information at least about the aircraft and an inventory management system that stores information about aircraft parts and tools used in maintenance activities, the maintenance module further comprising;

an initialization module for receiving login data generated based on interaction of the user with the audio recording device or the video recording device, the initialization module verifies the login data based on verification data in a datastore coupled to the initialization module whereupon once verified, configuration data about a particular aircraft and maintenance type is sent to the initialization module;

a part number determination module coupled to the initialization system for receiving part data wherein the part data is generated based on interaction of the user with the video recording device or the audio recording device wherein the part data comprises;

speech data of spoken commands of the user generated by the audio recording device wherein the speech data is processed using one or more speech processing techniques to determine a part number from the spoken commands of the user, and video data based on recorded video generated by the video recording device wherein the video data is processed using one more image processing techniques to determine a part number from a part number displayed in the recorded video, the part determination module performing a further step of confirming the part number based on aircraft information and maintenance type information;

an information processing module for receiving the part number to generate one or more requests for information associated with the part number by communicating the requests to the computing systems through a connection established via the initialization module, and in response to one or more requests, the information processing module receives part information generated by the computing systems for presenting by the display or audio device, the part information comprising;

a part option for selecting via the display or audio device an option to order a new part, to request of a maintenance procedure for a particular part, and to report the maintenance activity status to the computing systems, the information processing module communicates maintenance procedure data comprising;

the part option to the computing systems via the connection established by the initialization module.