Virtual world building operations center

• US 8,473,852 B2
• Filed: 07/28/2010
• Issued: 06/25/2013
• Est. Priority Date: 07/31/2009
• Status: Active Grant

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corp. (Siemens AG)
• Inventors: Russell, Steven A.
• Primary Examiner(s): Refai, Ramsey
• Assistant Examiner(s): STITT, ERIK V
• Application Number:
  US12/844,904
• Publication Number:
  US 20110029897A1
• Time in Patent Office: 1,063 Days
• Field of Search: 715/810, 715/764, 715/757
• US Class Current: 715/757
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house

Abstract

Methods and systems are described that allow coordinated users to employ avatars to observe and control a plurality of buildings—their building support systems and functions from a virtual world environment. Embodiments coordinate building supervisory personnel (specialists), building monitoring and control equipment, and repair (worker) personnel in a virtual world that is controlled by specialist avatars. Operations personnel perform building operation decisions in response to current situations in a virtual world using their specialist avatars.

Claims

1. A method for maintaining real world buildings using a virtual world environment comprising:

creating one or more virtual buildings each corresponding with a real world building, each virtual building comprises;

an exterior;

an interior;

exterior and interior building equipment; and

embedded building attributes; and

creating a Virtual Building Operations Center (VBOC) to manage the one or more virtual buildings, the VBOC comprises;

one or more specialist avatar consoles, each specialist avatar console corresponds with a specialist avatar activity and has a virtual computing device displaying information pertaining to the specialist avatars'"'"' activity;

one or more table 3D displays; and

one or more vertical displays; and

interfacing each of the one or more virtual buildings with their real world building and the VBOC;

creating a worker specialist avatar controlled to manage coordination of worker avatars and expert worker avatars according to a predefined responsibility, the worker specialist, expert worker and worker avatars have a first person or a third person Field of View (FOV), can walk or teleport to destinations, talk with other specialist or worker avatars and manipulate objects;

and

responding to a special condition for the one or more real world buildings from the VBOC;

comprising;

using a computing device, logging a real world worker responsible for performing real world building maintenance into the VBOC as a worker avatar, including rendering the worker avatar to capture physical motions of the real world worker, and presenting to the real world worker a work assignment pertaining to the special condition;

using a computing device, logging a real world expert worker responsible for providing expertise to the real world worker into the VBOC as an expert worker avatar including rendering the expert worker avatar to capture physical motions of the real world expert worker, and presenting to the real world expert worker the work assignment pertaining to the special condition;

teleporting the worker avatar and the expert worker avatar from the VBOC to a virtual world work site corresponding to a real world building work site experiencing the special condition;

capturing the physical motions of the real world worker at the real world work site using sensors;

detecting changes to the real world work site based on signals from real equipment at the real world work site;

causing worker avatar motions according to the physical motions of the real world worker, and causing changes to the virtual world work site according to the changes to the real world work site; and

providing guidance from the worker specialist avatar and from the expert worker avatar to the real world worker for performing the work assignment, the guidance being based at least in part on the worker avatar motions and the changes to the virtual world work site.

2. The method according to claim 1 wherein the specialist and worker avatars move throughout the VBOC and the one or more virtual buildings and respond to visual indications in the VBOC and the one or more virtual buildings.

3. The method according to claim 1 wherein the specialist avatars predefined responsibilities are regulatory and safety, equipment, incident, finance, worker and security.

4. The method according to claim 3 wherein the worker avatar mimics one of electrical, plumbing, HVAC and security workers.

5. The method according to claim 1 wherein interfacing the one or more virtual buildings with their real world building further comprises interrogating the one or more real world building control systems.

6. The method according to claim 1 wherein the one or more vertical displays further comprises displaying information from the one or more real world building control systems, external multimedia streams and data from external applications.

7. The method according to claim 1 wherein the one or more vertical displays further comprises displaying views from the one or more virtual buildings.

8. The method according to claim 1 wherein responding to one or more real world building special conditions further comprises connecting virtual objects that have a relationship with one or more pointers in the VBOC or in the one or more virtual buildings.

9. The method according to claim 8 wherein the pointers are visible lines in a specialist avatar'"'"'s or worker avatar'"'"'s FOV.

10. The method according to claim 9 further comprising linking the pointers to related virtual objects or to views of real world objects displayed on the one or more vertical displays, table 3D displays or virtual computing device displays.

11. The method according to claim 1 wherein interfacing the one or more virtual buildings with their real world buildings further comprises imbuing the one or more virtual world buildings with operational and informational attributes and behaviors so that the one or more virtual world buildings visually change when affected by a special condition.

12. The method according to claim 11 wherein a visual change reflects an abnormal building condition and/or includes explanatory text.

13. The method according to claim 1 wherein specialist or worker avatars further comprise touching a virtual world object to issue a command or to retrieve information.

14. The method according to claim 4 further comprising receiving in the VBOC a visual and audible special condition alarm from a virtual building requiring attention.

15. The method according to claim 14 wherein the alarm is a 3D object representing a corresponding real world building component accompanied by a scale model of the virtual building experiencing the problem.

16. The method according to claim 15 wherein the virtual building scale model appears on the one or more table 3D displays.

17. The method according to claim 16 further comprising:

acknowledging the special condition alarm by an incident specialist avatar using its virtual computing device to silence the audible alarm; and

displaying where the special condition has occurred on the virtual building scale model on the incident specialist avatar'"'"'s console.

18. The method according to claim 17 further comprising:

examining equipment responsible for the special condition by an equipment specialist avatar using its virtual computing device; and

reviewing the special condition and determining a course of action by the incident specialist avatar wherein the course of action may require one or more real world workers.

19. The method according to claim 18 wherein to determine a course of action further comprises leaving the VBOC and entering the virtual building experiencing the special condition by the equipment specialist avatar walking or teleporting.

27. The method according to claim 1 wherein for the real world worker at the real world work site and the worker avatar at the virtual work site, objects appear similar in form and layout except at the virtual work site the worker avatar can touch or manipulate an object to view information.

34. The method according to claim 1 further comprising controlling the one or more real world building operations from the VBOC.

20. The method according to 19 further comprising selecting one or more real world workers by the worker specialist avatar and contacting the selected one or more real world workers via telephony, email or instant message.

21. The method according to claim 20 further comprising tracking the one or more real world workers as they travel to the real world building work site experiencing the special condition on a 2D map or 3D map displayed on the worker specialist avatar'"'"'s virtual computing device or on the one or more table 3D displays.

22. The method according to claim 21 wherein the one or more real world workers are tracked by GPS, mobile phone position, radio triangulation or by verbal reports.

23. The method according to claim 22 further comprising:

using a computing device at the real world work site, viewing a first worker avatar'"'"'s work site FOV of structures and equipment with data displays of hazards or regulations; and

having the first worker avatar touch objects to bring into view additional information.

24. The method according to claim 23 wherein the additional information includes work flows, safety guidelines, visible dynamic attributes that indicate abnormal operating conditions, 3D models of the structures and equipment, exploded-view models, and repair history.

25. The method according to claim 23 wherein capturing the physical motions of the real world worker at the real world work site comprises detecting the motions from pedometers, gyroscopic acceleration detectors, or the worker specialist avatar manipulating the one or more worker avatars to match their physical motions.

26. The method according to claim 25 further comprising:

supplying information from the real world worker to the equipment specialist avatar using telephony for verbal messages, photographs or text messages; and

displaying the real world worker information on the worker specialist avatar'"'"'s virtual computing device or on the one or more vertical displays.

28. The method according to claim 23 wherein if a second worker avatar or expert worker avatar is present at the virtual work site, the second worker avatar or expert worker avatar may or may not be authorized to view the same information as the first worker avatar due to job role.

29. The method according to claim 23 wherein if a second worker avatar is present at the virtual work site, the first and second worker avatars can communicate with each other using telephony, VOIP or text messages.

30. The method according to claim 26 wherein information from the real world work site is displayed in the VBOC with selected information shown to a financial specialist avatar that is familiar with equipment costs, personnel actions and estimated lost revenues.

31. The method according to claim 26 wherein information from the real world work site is displayed in the VBOC with selected information shown to a regulation-green specialist avatar that is familiar with regulations and sustainability goals.

32. The method according to claim 25 further comprising modeling the worker avatar'"'"'s actions to produce best-practice sequences.

33. The method according to claim 25 further comprising after the real world work site work is completed, closing out a work order and storing the worker avatar'"'"'s work sequences in the form of data on avatar and worker motions, voice and video recordings, 3D machinima recordings and annotations for later analysis.