System and method for carrying out and visually displaying simulations in an augmented reality

• US 20070088526A1
• Filed: 11/03/2004
• Published: 04/19/2007
• Est. Priority Date: 11/10/2003
• Status: Active Grant

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Moritz, Soeren, Schlereth, Michael, Friedrich, Wolfgang
• Granted Patent: US 7,852,355 B2
• Time in Patent Office: Days
• US Class Current: 702/150
• CPC Class Codes: G06F 3/012 Head tracking input arrangements

Abstract

The invention relates to a system and method inside an augmented reality system for visually displaying simulation results in a mixed virtual-real environment. The system and method permit one or more users to carry out simulation processes in the context of a real environment, particularly in the field of industrial automation systems and to visually display their static and dynamic results in the contest of the real environment. Processes running in the real environment are detected and synchronized with the simulation. A control unit enables a reciprocal influencing of real processes with the simulation. In addition, the user can control the execution of the simulation via a user interface.

Claims

1-18. -18. (canceled)

19. A system for presenting information to at least one user, comprising:

at least one recording unit for scanning an environment and for generating corresponding environment information identifying a position or an orientation of the system relative to the environment;

at least one simulation system for generating simulation data; and

at least one processing unit configured to;

continuously adapt image data stored in a first memory based upon the simulation data; and

link the environment information to the adapted image data.

20. The system in accordance with claim 19, wherein the processing unit is further configured to:

calculate concealments of virtual objects corresponding to concealments of real objects arranged in a recording range of the system based upon the image data; and

generate a volume data set representing the virtual objects, wherein such surfaces of the virtual objects corresponding to concealed surfaces of the real objects are hidden.

21. The system in accordance with claim 20, further comprising at least one display unit for displaying the volume data set.

22. The system in accordance with claim 19, further comprising at least one application controller having an interface for controlling the simulation system.

23. The system in accordance with claim 22, wherein the application controller is further configured to interact with a real process of the environment via the interface.

24. The system in accordance with claim 23, further comprising a second memory for holding current status values acquired from the real process.

25. The system in accordance with claim 24, wherein the status values include sensor values or actuator setpoint values.

26. The system in accordance with claim 24, wherein a simulation sequence created by the simulation system is continuously adapted based on the status values.

27. The system in accordance with claim 24, further comprising at least one process link to the real process such that a status of the real process can be changed by the system via the interface.

28. The system in accordance with claim 19, further comprising a third memory having reconstruction data, the reconstruction data configured to reconstruct a real process simulated by the simulation system.

29. A method of presenting information to at least one user, comprising:

scanning an environment by at least one recording unit;

generating corresponding environment information identifying a position or an orientation of the system relative to the environment, by the recording unit;

generating simulation data by at least one simulation system;

continuously adapting image data stored in a first memory based upon the simulation data, by at least one processing unit; and

linking the environment information to the adapted image data, by the processing unit.

30. The method in accordance with claim 29, further comprising:

calculating concealments of virtual objects corresponding to concealments of real objects arranged in a recording range of the system based upon the image data, by the processing unit; and

generating a volume data set representing the virtual objects, by the processing unit, wherein such surfaces of the virtual objects corresponding to concealed surfaces of the real objects are hidden.

31. The method in accordance with claim 30, further comprising displaying the volume data set on a display.

32. The method in accordance with claim 29, further comprising interfacing with the simulation system or with a real process of the environment, by an application controller having an interface.

33. The method in accordance with claim 32, further comprising storing current status values of the real process in a second memory.

34. The method in accordance with claim 33, further comprising continuously adapting a simulation sequence created by the simulation system based on the status values.

35. The method in accordance with claim 33, further comprising modifying a status of the real process based on the status values.

36. The method in accordance with claim 29, further comprising reconstructing a real process simulated by the simulation process based on reconstruction data stored in a third memory.