Maintenance assistant system

• US 9,530,057 B2
• Filed: 11/26/2013
• Issued: 12/27/2016
• Est. Priority Date: 11/26/2013
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Conner, Kevin J, Mathan, Santosh
• Primary Examiner(s): Jebari, Mohammed
• Application Number:
  US14/090,106
• Publication Number:
  US 20150146008A1
• Time in Patent Office: 1,127 Days
• US Class Current: 1/1
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09B 9/006 for locating or ranging of objects

Abstract

A maintenance assistance system and method of operating are provided. The maintenance assistance system may include, but is not limited to, a camera, a speaker, a technician marker, a memory configured to store a location of a plurality of tags and a location of at least one component relative to at least one of the plurality of tags, and a processor communicatively coupled to the camera, the speaker and the memory, the processor configured to receive an indication of one of the components, determine a location of the technician marker relative to the location of the indicated component based upon data from the camera and the location of the indicated component relative to the at least one of the plurality of tags, and output position information to the speaker based upon the determined location of the technician marker relative to the location of the indicated component.

Claims

1. A maintenance assistance system, comprising:

a camera;

a speaker;

a wearable technician marker;

a memory configured to store a location of each of a plurality of tags and a location of at least one serviceable component relative to at least one of the plurality of tags; and

a processor communicatively coupled to the camera, the speaker and the memory, the processor configured to;

receive an indication of a component to be serviced from the at least one serviceable components;

determine a location of the indicated component to be serviced relative to the locations of at least one of the plurality of tags stored in the memory based upon a distance of the indicated component to be serviced from the at least one of the plurality of tags and an angle between the indicated component to be serviced and the at least one of the plurality of tags;

determine a location of the wearable technician marker relative to the determined location of the indicated component to be serviced based upon data from the camera including a distance of the wearable technician marker from the indicated component to be serviced and an angle between the wearable technician marker and the indicated component to be serviced; and

output position information comprising an instruction to locate the indicated component to be serviced relative to the wearable technician marker to the speaker based upon the determined location of the wearable technician marker relative to the location of the indicated component to be serviced.

2. The maintenance assistance system of claim 1, wherein the position information includes an affirmation that the wearable technician marker is within a predetermined distance of the indicated component to be serviced.

3. The maintenance assistance system of claim 1, wherein the wearable technician marker is on a glove.

4. The maintenance assistance system of claim 1, wherein the processor is further configured to determine a location of the wearable technician marker relative to the location of the indicated component to be serviced by:

determining the location of the indicated component to be serviced relative to at least one of the plurality of tags via triangulation; and

comparing the determined location of the indicated component to be serviced to a position of the wearable technician marker.

5. The maintenance assistance system of claim 1, wherein the processor is further configured to output the location of the indicated component to be serviced to the speaker based upon the location of the indicated component to be serviced stored in the memory.

6. The maintenance assistance system of claim 1, wherein the processor is further configured to:

receive a request for a maintenance procedure from an input system; and

output instructions for the maintenance procedure to the speaker.

7. The maintenance assistance system of claim 6, wherein the processor is further configured to store data from at least one of the camera and the input system corresponding to an execution of the maintenance procedure in the memory.

8. A method for operating a maintenance assistance system comprising a camera, a wearable technician marker, a speaker, a memory and a processor communicatively coupled to the camera, the speaker and the memory, the method comprising:

receiving, by the processor, an indication of a component to be serviced;

determining, by the processor, a location of the indicated component to be serviced relative to the locations of at least one of the plurality of tags stored in the memory based upon a distance of the indicated component to be serviced from the at least one of the plurality of tags and an angle between the indicated component to be serviced and the at least one of the plurality of tags;

determining, by the processor, a location of the wearable technician marker relative to the location of the indicated component to be serviced based upon data from the camera including a distance of the wearable technician marker from the indicated component to be serviced and an angle between the wearable technician marker and the indicated component to be serviced and a location of the indicated component relative to at least one tag stored in the memory; and

outputting position information comprising instructions to locate the indicated component to be serviced relative to the wearable technician marker to the speaker based upon the determined location of the wearable technician marker relative to the location of the indicated component to be serviced.

9. The method of claim 8, wherein the determining further comprises:

determining, by the processor, the location of the indicated component to be serviced relative to the at least one tag via triangulation; and

comparing, by the processor, the determined location of the indicated component to be serviced to a position of the wearable technician marker.

10. The method of claim 8, further comprising outputting, by the processor, the location of the indicated component to be serviced to the speaker based upon a location of the indicated component to be serviced stored in the memory.

11. The method of claim 8, further comprising:

receiving, by the processor, a request for maintenance procedure instructions; and

outputting, by the processor, the maintenance procedure instructions to the speaker.

12. The method of claim 8, further comprising:

recording, by the camera, a technician executing a maintenance procedure; and

storing the recording in the memory.

13. A non-transitory computer-readable medium storing instructions, which when executed by a processor, operate a maintenance assistance system comprising the processor, a camera, a wearable technician marker, a speaker, and a memory, the instructions causing the processor to:

receive an indication of a component to be serviced;

determine a location of the indicated component to be serviced relative to the locations of at least one of the plurality of tags stored in the memory based upon a distance of the indicated component to be serviced from the at least one of the plurality of tags and an angle between the indicated component to be serviced and the at least one of the plurality of tags;

determine a location of the wearable technician marker relative to the location of the indicated component to be serviced based upon data from the camera including a distance of the wearable technician marker from the indicated component to be serviced and an angle between the wearable technician marker and the indicated component to be serviced and a location of the indicated component relative to at least one tag stored in the memory; and

output position information comprising instructions to locate the indicated component to be serviced relative to the wearable technician marker to the speaker based upon the determined location of the wearable technician marker relative to the location of the indicated component to be serviced.

14. The non-transitory computer-readable medium of claim 13 wherein the instructions further cause the processor to:

determine the location of the indicated component to be serviced relative to the at least one tag via triangulation; and

compare the determined location of the indicated component to be serviced to a position of the wearable technician marker.

15. The non-transitory computer-readable medium of claim 13 wherein the instructions further cause the processor to output the location of the indicated component to be serviced to the speaker based upon a location of the indicated component to be serviced stored in the memory.

16. The non-transitory computer-readable medium of claim 13 wherein the instructions further cause the processor to:

receive a request for maintenance procedure instructions; and

output the maintenance procedure instructions to the speaker.

17. The non-transitory computer-readable medium of claim 16 wherein the instructions further cause the processor to

record a technician executing a maintenance procedure via the camera;

and

store the recording in the memory.