Apparatus and method of displaying hardware status using augmented reality

• US 9,965,564 B2
• Filed: 07/26/2011
• Issued: 05/08/2018
• Est. Priority Date: 07/26/2011
• Status: Active Grant

• Current Assignee: Schneider Electric IT Corporation (Schneider Electric SE)
• Original Assignee: Schneider Electric IT Corporation (Schneider Electric SE)
• Inventors: Whelihan, Michael L.
• Primary Examiner(s): Stanford, Christopher
• Application Number:
  US13/190,688
• Publication Number:
  US 20130026220A1
• Time in Patent Office: 2,478 Days
• Field of Search: 235375, 235385
• CPC Class Codes: G06F 16/9554 by using bar codes
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A method of accessing equipment status information in a data center includes detecting, by a mobile computing device, a fiduciary marker within at least one image captured by an image capture device, decoding the equipment identification data from the fiduciary marker, providing the equipment identification data to a data center management system for the data center through a communications network, receiving status information for the piece of equipment from the data center management system through the communication network, and displaying the status information on a display screen of the mobile computing device. The fiduciary marker is configured to contain encoded equipment identification data associated with a piece of equipment.

Claims

1. A method of accessing equipment status information in a data center, the method comprising:

detecting within the data center, by a mobile computing device, a first fiduciary marker affixed on or near a first piece of equipment within at least one image displayed on a display screen of the mobile computing device, the at least one image captured by an image capture device coupled to the mobile computing device, the first fiduciary marker configured to contain encoded first equipment identification data associated with the first piece of equipment;

decoding, by the mobile computing device, the first equipment identification data from the first fiduciary marker;

providing, by the mobile computing device, the first equipment identification data to a data center management system for the data center through a communications network;

receiving, by the mobile computing device, status information that notifies a user of at least one of an alert and an alarm active for the first piece of equipment from the data center management system, responsive to providing the first equipment identification data to the data center management system, through the communication network; and

superimposing the status information upon the at least one image displayed on the display screen of the mobile computing device.

2. The method of claim 1, wherein the first fiduciary marker includes a two-dimensional barcode.

3. The method of claim 1, wherein detecting within the data center, by a mobile computing device, further includes capturing a field of view of the image capture device that includes a video feed, the field of view including all visible objects and the first piece of equipment.

4. The method of claim 1, wherein superimposing the status information includes superimposing status information for the first piece of equipment, and further comprising sending, by the mobile computing device and through the communication network, a control command to the first piece of equipment.

5. The method of claim 1, wherein the first equipment identification data includes an Internet Protocol (IP) address of the first piece of equipment.

6. The method of claim 1, wherein the first equipment identification data includes a serial number of the first piece of equipment.

7. The method of claim 1, wherein the first equipment identification data includes a Media Access Control (MAC) address of the first piece of equipment.

8. The method of claim 1, wherein the first equipment identification data includes a hostname associated with the first piece of equipment.

9. The method of claim 1, wherein the first equipment identification data includes a model number of the first piece of equipment.

10. The method of claim 1, wherein the first equipment identification data includes a location of the first piece of equipment.

11. The method of claim 1, further comprising selecting the status information via a GUI, and receiving and displaying additional information.

12. The method of claim 1, wherein the at least one image includes a live video feed.

13. An apparatus for accessing equipment status information in a data center, the apparatus comprising:

a camera; and

a mobile computing device having a processor, a display screen and a wireless communication unit, the mobile computing device coupled to the camera and configured to receive at least one image captured by the camera;

the processor configured to;

detect a first fiduciary marker within the at least one image, the first fiduciary marker affixed on or near a first piece of equipment and configured to contain encoded first equipment identification data associated with the first piece of equipment in the data center;

decode the first equipment identification data from the first fiduciary marker;

send the first equipment identification data to a data center management system for the data center using the wireless communication unit;

receive first status information that notifies a user of at least one of an alert and an alarm active for the first piece of equipment from the data center management system, responsive to the first equipment identification data sent to the data center management system and decoded from the first fiduciary marker, using the wireless communication unit; and

superimpose the first status information upon the at least one image displayed on the display screen.

14. The apparatus of claim 13, wherein the first fiduciary marker includes a two-dimensional barcode.

15. The apparatus of claim 13, wherein the at least one image includes a field of view of the image capture device, the field of view including all visible objects and the first piece of equipment.

16. The apparatus of claim 13, wherein the processor is further configured to superimpose the first status information and to send a control command to the first piece of equipment using the wireless communication unit.

17. The apparatus of claim 13, wherein the first equipment identification data includes at least one of an Internet Protocol (IP) address of the first piece of equipment, a serial number of the first piece of equipment, a Media Access Control (MAC) address of the first piece of equipment, a hostname associated with the first piece of equipment, a model number of the first piece of equipment, and a location of the first piece of equipment.

18. The apparatus of claim 13, wherein the processor is further configured to superimpose the first status information and to send a request for additional information pertaining to the first status information.

19. A computer readable medium comprising computer-executable instructions that when executed on a processor perform the acts of:

detecting, within a data center, a first fiduciary marker within at least one image displayed on a display screen coupled to the processor, the at least one image captured by an image capture device coupled to the processor, the first fiduciary marker affixed on or near a first piece of equipment and configured to contain encoded first equipment identification data associated with the first piece of equipment;

decoding the first equipment identification data from the first fiduciary marker;

providing the first equipment identification data to a data center management system for the data center through a communications network;

receiving first status information that notifies a user of at least one of an alert and an alarm active from the data center management system, responsive to the first equipment identification data provided to the data center management system, through the communication network; and

superimposing the first status information upon the at least one image displayed on the display screen coupled to the processor.

20. The computer readable medium of claim 19, wherein the first fiduciary marker includes a two-dimensional barcode having hardware identification data encoded therein.