Systems and methods for immersive interaction with actual and/or simulated facilities for process, environmental and industrial control

• US 8,594,814 B2
• Filed: 06/19/2009
• Issued: 11/26/2013
• Est. Priority Date: 06/20/2008
• Status: Active Grant

• Current Assignee: Invensys Systems Incorporated (Invensys PLC)
• Original Assignee: Invensys Systems Incorporated (Invensys PLC)
• Inventors: Rovaglio, Maurizio, Scheele, Tobias
• Primary Examiner(s): Padmanabhan, Kavita
• Assistant Examiner(s): LEE, DOUGLAS S
• Application Number:
  US12/487,792
• Publication Number:
  US 20090319058A1
• Time in Patent Office: 1,621 Days
• Field of Search: 700/83, 700/17, 700/97, 700/98, 715848-852, 345418-421
• US Class Current: 700/83
• CPC Class Codes: G05B 17/02 electric
• CPC Class Codes: G05B 2219/31018 Virtual factory, modules in network, can be selected and combined at will
• CPC Class Codes: G05B 23/0216 Human interface functionality, e.g. monitoring system providing help to the user in the selection of tests or in its configuration
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: Y02P 90/80 Management or planning

Abstract

The invention provides, in some aspects, systems for interaction with a control environment that includes controlled equipment along with control devices that monitor and control that controlled equipment. According to some of those aspects, such a system includes first functionality that generates output representing an operational status of the controlled equipment, as well as second functionality that generates output representing an operational of one or more of the control devices. An engine coordinates the first functionality and to the second functionality to generate an operational status of the control environment. A virtual reality environment generates, as a function of that operational status and one or more physical aspects of the control environment, a three-dimensional (“3D”) display of the control environment. The virtual reality environment is responsive to user interaction with one or more input devices to generate the 3D display so as to permit the user to interact with at least one of the control devices and the controlled equipment at least as represented by the 3D display of the control environment. The engine applies to at least one of the first and second functionality indicia of those interactions to discern resulting changes in the operational status of the control environment. It applies indicia of those changes to the virtual reality environment to effect corresponding variation in the 3D display of the control environment—i.e., variation indicative of the resulting change in the control environment.

Claims

1. A system having a control environment that includes controlled equipment and control devices that monitor and control that controlled equipment, the system including:

first functionality generating output representing an operational status of one or more of said controlled equipment,

second functionality generating output representing an operational status of one or more of said control devices,

an engine, coupled to the first functionality and to the second functionality, that generates an operational status of the control environment, wherein said engine includes simulation functionality, the simulation functionality comprising dynamic simulation software,

a virtual reality environment, coupled to the engine and to one or more representations of one or more physical aspects of the control environment, the virtual reality environment generating a three-dimensional (“

3D”

) display of the control environment,

the virtual reality environment being responsive to user interaction with one or more input devices to generate the 3D display of the control environment of the control environment so as to permit the user to interact with at least one of the control devices and the controlled equipment at least as represented by the 3D display of the control environment,

the engine applying to at least one of the first and second functionality indicia of user interaction with at least one of the control devices and the controlled equipment to discern a resulting change in the operational status of the control environment, and the control engine applying indicia of such change to the virtual reality environment to effect corresponding variation in the 3D display of the control environment, and;

wherein the first functionality includes a first simulation functionality that simulates the operation of one or more of the controlled devices and the second functionality includes a second simulation functionality that simulates operation of one or more the control devices, the input for the engine is said indicia.

2. The system of claim 1, wherein the virtual reality environment is coupled to a database comprising, as one or more representations of one or more physical aspects of the control, environment, a three-dimensional laser scan of at least a portion of the control environment including (i) on or more controlled equipment, (ii) one or more control devices.

3. The system of claim 1, wherein the virtual reality environment is coupled to a database comprising, as one or more representations of one or more physical aspects of the control environment, a plurality of two-dimensional images of at least a portion of the control environment including (i) on or more controlled equipment, (ii) one or more control devices.

4. The system of claim 3, wherein the virtual reality environment is coupled to functionality that generates a three-dimensional representation of the control environment from the plurality of two-dimensional images.

5. The system of claim 1, wherein the virtual reality environment includes a database comprising, as one or more representations of one or more physical aspects of the control environment, a computer-aided design/computer-aided manufacturing (CAD/CAM) representation of at least a portion of the control environment including (i) on or more controlled equipment, (ii) one or more control devices.

6. The system of claim 1, adapted to permit a user to experience, via the 3D display of the control environment, interaction with and travel through the control environment.

7. The system of claim 1, wherein the virtual reality environment augments the 3D display of the control environment by any of (i) superimposing graphics on representational displays of one or more controlled equipment and/or one or more control devices, and (ii) altering the representation of one or more of controlled equipment and/or one or more of the control devices to give the user the impression of seeing into, through and/or around objects.

8. The system of claim 7, wherein the virtual reality environment augments the 3D display of the control environment by superimposing graphics, including any of graphs, equations, still images, video images and other visual aids, on displays of one or more controlled equipment and/or one or more control devices.

9. The system of claim 7, wherein the virtual reality environment augments the 3D display of one or more controlled equipment and/or one or more control devices affected by the actions of the user.

10. The system of claim 7, wherein the virtual reality environment augments the 3D display of the control environment based on one or more of (i) simulated and/or actual operational data for one or more of the controlled equipment and/or one or more of the control devices, and (ii) still and/or video images acquired any of in or around the control environment.

11. The system of claim 1, wherein one or more of the control devices comprise actuator-type field devices, sensor-type field devices, interface devices and/or controllers.

12. The system of claim 1, wherein the first simulation function is dynamic simulation.

13. The system of claim 2, wherein the second simulation function is dynamic simulation.

14. A system having a control environment that includes controlled equipment and control devices that monitor and control that controlled equipment, the system including:

first functionality generating output representing an operational status of a plant or other environment that includes one or more of said controlled equipment,

second functionality generating output representing an operational status of one or more of said control devices,

one or more databases coupled to any of the first functionality and the second functionality, the one or more databases including any of configuration and a modeling database (“

configuration”

database), an application database and a runtime database,

an engine, coupled to the first functionality and to the second functionality, that generates an operational status of the control environment, wherein said engine includes simulation functionality, the simulation functionality comprising dynamic simulation software,

a virtual reality environment, coupled to the engine and to one or more representations of one or more physical aspects of the control environment, the virtual reality environment generating a three-dimensional display of the control environment,

the virtual reality environment being responsive to user interaction with one or more input devices to generate the 3D display of the control environment so as to permit the user to interact with at least one of the control devices and the controlled equipment at least as represented by the 3D display of the control environment,

the engine applying to the at least one of the first and second functionality indicia of user interaction with at least one of the control devices and the controlled equipment to discern changes on the operational status of the control environment, and the control engine applying indicia of those changes to the virtual reality environment to effect variation in the 3D display of the control environment, and;

wherein the first functionality includes a first simulation functionality that simulates the operation of one or more of the controlled devices and the second functionality includes a second simulation functionality that simulates operation of one or more the control devices, the input for the engine is said indicia.

15. The system of claim 14, wherein the configuration database contains one or more control strategies for one or more of the controlled equipment and/or one or more of the control devices.

16. The system of claim 14, wherein one or more of the databases includes simulated and/or actual current operational data for one or more of the controlled equipment and/or one or more of the control devices and/or one or more of the controlled equipment.

17. The system of claim 14, wherein one or more of the databases includes simulated and/or actual historical operational data for the control devices and/or one or more of the controlled equipment.

18. The system of claim 14, in which all or parts of the control environment may be any of actual or simulated.

19. The system of claim 14, wherein the first functionality comprises dynamic simulation functionality that simulates operation of one or more of the controlled equipment.

20. The system of claim 19, wherein the dynamic simulation functionality comprises a solver or other system for solving mathematical systems of equations and/or to other simulate operation of a control environment.

21. The system of claim 14, wherein the second functionality comprises software or other functionality that simulates operation of one or more of the control devices.

22. The system of claim 14, wherein one or more of the control devices comprise actuator-type field devices, sensor-type field devices, interface devices and/or controllers.

23. A system having a control environment that includes controlled equipment and control devices that monitor and control that controlled equipment, the system including:

first functionality generating output representing an operational status of a plant or other environment that includes one or more of said controlled equipment,

second functionality generating output representing an operational status of one or more of said control devices,

one or more databases coupled to any of the first functionality and the second functionality, the one or more databases including any of configuration and a modeling database (“

configuration”

database), an application database and a runtime database,

an engine, coupled to the first functionality and to the second functionality, that generates an operational status of the control environment, wherein said engine includes simulation functionality, the simulation functionality comprising dynamic simulation software,

a virtual reality environment, coupled to the engine and to one or more representations of one or more physical aspects of the control environment, the virtual reality environment generating a three-dimensional display of the control environment,

the virtual reality environment being responsive to user interaction with one or more input devices to generate the 3D display of the control environment so as to permit the user to interact with at least one of the control devices and the controlled equipment at least as represented by the 3D display of the control environment,

the engine applying to the at least one of the first and second functionality indicia of user interaction with at least one of the control devices and the controlled equipment to discern changes on the operational status of the control environment, and the control engine applying indicia of those changes to the virtual reality environment to effect variation in the 3D display of the control environment,

a control room digital data processor, coupled to any of the engine and the virtual reality environment, that monitors and/or controls user interaction with at least one of the control devices and the controlled equipment at least as represented by the 3D display of the control environment, and;

wherein the first functionality includes a first simulation functionality that simulates the operation of one or more of the controlled devices and the second functionality includes a second simulation functionality that simulates operation of one or more the control devices, the input for the engine is said indicia.

24. The system of claim 23, wherein the control room digital data processor is adapted to enable users thereof to witness effects of interaction between users of the virtual reality environment with respect to at least one of the control devices and the controlled equipment at least as represented by the 3D display of the control environment.

25. The system of claim 24, wherein the virtual reality environment is adapted to enable users thereof to witness effects of actions by users of the control room digital data processor.

26. The system of claim 23, wherein one or more of the control devices comprise actuator-type field devices, sensor-type field devices, interface devices and/or controllers.

27. A method of interaction with a control environment that includes controlled equipment and control devices that monitor and control that controlled equipment, the method comprising:

generating first functionality output representing an operational status of one or more of said controlled equipment, generating second functionality output representing an operational status of one or more of said control devices, an engine, coupled to the first functionality and to the second functionality, that generates an operational status of the control environment, wherein said engine includes simulation functionality, the simulation functionality comprising dynamic simulation software, a virtual reality environment, coupled to the engine and to one or more representations of one or more physical aspects of the control environment, the virtual reality environment generating a three-dimensional (“

3D”

) display of the control environment, the virtual reality environment being responsive to user interaction with one or more input devices to generate the 3D display of the control environment of the control environment so as to permit the user to interact with at least one of the control devices and the controlled equipment at least as represented by the 3D display of the control environment, the engine applying to at least one of the first and second functionality indicia of user interaction with at least one of the control devices and the controlled equipment to discern a resulting change in the operational status of the control environment, and the control engine applying indicia of such change to the virtual reality environment to effect corresponding variation in the 3D display of the control environment, and;

wherein the first functionality includes a first simulation functionality that simulates the operation of one or more of the controlled devices and the second functionality includes a second simulation functionality that simulates operation of one or more the control devices, the input for the engine is said indicia.

28. The method of claim 27, wherein step (A) includes utilizing one or more databases including any of configuration and a modeling database (“

configuration”

database), an application database and a runtime database to determine any of an operational status of the one or more of controlled equipment and the one or more control devices.

29. The method of claim 27, comprising

tracking positions of one or more persons with respect to the control environment at least as represented by the 3D display thereof, and wherein

the generating step include generating the 3D display to indicate one or more of those positions of those persons.

30. The method of claim 27, wherein the generating step includes augmenting the 3D display of the control environment by any of (i) superimposing graphics on representational displays of one or more controlled equipment and/or one or more control devices, and (ii) altering the representation to give the user the impression of seeing into, through and/or around objects.