System and method for carrying out and visually displaying simulations in an augmented reality

• US 7,852,355 B2
• Filed: 11/03/2004
• Issued: 12/14/2010
• Est. Priority Date: 11/10/2003
• Status: Active Grant

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Moritz, Soeren, Schlereth, Michael, Friedrich, Wolfgang
• Primary Examiner(s): GOOD JOHNSON, MONTILEWA
• Application Number:
  US10/578,940
• Publication Number:
  US 20070088526A1
• Time in Patent Office: 2,232 Days
• Field of Search: 345/633, 345/632
• US Class Current: 345/633
• CPC Class Codes: G06F 3/012 Head tracking input arrangements

Abstract

The invention relates to a system and method inside an augmented reality system for visually displaying simulation results in a mixed virtual-real environment. The system and method permit one or more users to carry out simulation processes in the context of a real environment, particularly in the field of industrial automation systems and to visually display their static and dynamic results in the contest of the real environment. Processes running in the real environment are detected and synchronized with the simulation. A control unit enables a reciprocal influencing of real processes with the simulation. In addition, the user can control the execution of the simulation via a user interface.

Claims

1. A method for performing a simulation in an industrial automation system for observation by at least one user, comprising:

placing a real component of the industrial automation system under the control of a process controller, the real component having a process setting determinable by input of a process value via a process link;

providing a process interface in the automation system for receiving input from the process link and connected for setting the real component in accord with the process value;

controlling the real component via the process controller and the process link, wherein the real component forms part of a real environment; and

providing a mixed virtual/real environment for presentation to the user of a dynamic simulation in a context which includes the real component in the real environment by augmenting the real environment with a simulated dynamization, thereby providing the user with a dynamic simulation in the context of the real environment, wherein ongoing processes running in the real environment are recorded and synchronized with the dynamic simulation, and wherein execution of the dynamic simulation is controllable by the user.

2. The method in accordance with claim 1, wherein the mixed virtual/real environment is displayed to the user by generating a volume data set representing the virtual objects and the real component.

3. The method in accordance with claim 2, wherein while under the control of the process controller, the real component receives actuator values having a process setting determinable by input of a process value via a process link, and provides one or more sensor values associated with one or more states of the component during operation, the method further including:

storing the sensor and actuator values for the real component; and

for provision of the mixed virtual/real environment with the simulated dynamization, providing the sensor and actuator values to perform modifications to the ongoing simulated dynamization.

4. The method in accordance with claim 1, further including recording data during the ongoing simulated dynamization to enable playback of the recorded dynamization in slow motion and in a backwards direction in time.