Distributed Physics Based Training System and Methods

• US 20090099824A1
• Filed: 11/28/2006
• Published: 04/16/2009
• Est. Priority Date: 11/28/2005
• Status: Active Grant

• Current Assignee: CAE USA Incorporated (CAE Incorporated)
• Original Assignee: L-3 Communications Corporation (L3Harris Technologies, Inc.)
• Inventors: Williams, Jon, Chen, Yushan, Pollak, Eytan, Falash, Mark
• Granted Patent: US 8,645,112 B2
• Time in Patent Office: Days
• US Class Current: 703/8
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A distributed simulation system is composed of simulator stations linked over a network that each renders real-time video imagery for its user from scene data stored in its data storage. The simulation stations are each connected with a physics farm that manages the virtual objects in the shared virtual environment based on their physical attribute data using physics engines, including an engine at each simulation station. The physics engines of the physics farm are assigned virtual objects so as to reduce the effects of latency, to ensure fair fight requirements of the system, and, where the simulation is of a vehicle, to accurately model the ownship of the user at the station. A synchronizer system is also provided that allows for action of simulated entities relying on localized closed loop controls to cause the entities to meet specific goal points at specified system time points.

Claims

1. A simulation station for use in a simulation system for a plurality of users each associated with a respective simulation station, said simulation station comprising:

a link to a communication network to which the simulation stations are linked;

computer accessible data storage storing scene data, the scene data including object data defining a plurality of virtual objects in a virtual environment, said object data including, for each of said virtual objects, location data defining a location of the virtual object in the virtual environment;

an image generator periodically rendering from said scene data a frame of video imagery corresponding to a view of the virtual environment from a position associated with the user of the simulation station so that the user is provided with a real time view of the virtual environment; and

a display receiving the video imagery and displaying said imagery so as to be viewable by the associated user;

a physics engine managing physics data for the virtual objects in the scene data by determining physics data for a subset of the virtual objects in the scene based on physical attribute data defining virtual physical parameters of each of said virtual objects and physical rules of motion of the physics engine, and receiving, via the link, physics data regarding other of the virtual objects in the scene data;

said physics engine being connected with the data storage and influencing said scene data so that the location data of the virtual objects as defined in the scene data comply with the physical rules governing, based at least in part on the physical attribute data thereof, the movement thereof in the virtual environment.

2. The simulator station according to claim 1, and said physics data comprising position data indicative of a position of the corresponding object, force data indicative of accelerations to which the object is subjected, and time data indicating a time when values of said position data and said force data were current.

3. The simulator station according to claim 2, wherein, based on the data received for an object, the physics engine computes position of the object.

4. The simulator station according to claim 3, wherein the computed position relies on the physical rules of the physics engine and the physical attribute data for the object, wherein the determination relies on a projection of a path of the object caused by the acceleration acting thereon from the position in the physics data and any physical conflicts with any other objects in said path to the computed position.

5. The simulator station according to claim 4, wherein the physics engine receives a subsequent transmission of physics data corresponding to the object, and the physics engine causes the scene data to be updated so as to reflect the position of the object defined by the subsequent transmission of physics data.

6. The simulator station according to claim 5, wherein the scene data is updated incrementally to smooth movement of the object as rendered in the imagery and viewed by the user.

7. The simulator station according to claim 1, wherein the simulator station functions to simulate a real vehicle for the user, and wherein the scene data includes own-vehicle object data for an own-vehicle set of virtual objects that emulate the assembly of the real vehicle;

said simulator system simulating operation of the vehicle dynamically using the own-vehicle object data.

8. The simulator station according to claim 7, wherein the physics data for the own-vehicle set of objects is determined by the physics engine.

9. The simulator station according to claim 1, and said physics engine transmitting over the link physics data for a first virtual object in said subset of the virtual objects responsive to a determination that a change of contact of said virtual object with at least one of the other virtual objects has occurred.

10. The simulator station according to claim 9, wherein the physics data transmitted over the link includes entity identification data that identifies the first virtual object that has had a change of contact, colliding identification data that identifies the virtual object contacted, time data identifying the time of contact in system time, position data identifying a location of the contact, and force data corresponding to forces acting on the first virtual object as a result of said contact.

11. The simulator station according to claim 1, wherein the physics engine defines positions of objects using a first system of spatial coordinates, and wherein said the scene data defines locations of virtual objects using a second different system of spatial coordinates, said first system of coordinates being converted to said second system of coordinates when the physics engine influences locations of objects defined in the scene data.

12. The simulator station according to claim 11, wherein the second system of coordinates is GCC.

13. The simulator station according to claim 1, wherein a synchronizer at said simulator station receives data over the network, said data defining a goal in the virtual environment for a client virtual object to be completed by a point in time, said synchronizer coacting with the physics engine so as to cause the client virtual object to complete the goal by the point in time.

14. The simulator station according to claim 13, wherein the goal defined by the data includes a location in the virtual environment to which the client virtual object is to move to by said point in time.

15. The simulator station according to claim 14, wherein the data includes data defining a rate at which the client virtual object is to move to said location.

16. The simulator station according to claim 14, wherein the synchronizer uses a closed loop control process to prepare data transmitted to the physics engine so as to cause the client virtual object to move toward the location.

17. The simulator station according to claim 14, wherein, if obstruction of movement of the client virtual object by an obstructing object or objects is detected by the synchronizer, the synchronizer takes action to ensure that the goal is completed by the point in time, said action comprising causing the virtual client object to move toward the goal by a route that is not obstructed, or suspending at least an aspect of the physics rules with respect to the virtual client object and/or the obstructing object or objects so that the physics engine does not obstruct the movement of the client virtual object due to conflict with said obstructing object or objects.

18. A system for providing a shared simulated environment to a plurality of users each having a respective user location in said simulated environment, said system comprising:

a plurality of simulator stations each associated with a respective user and linked over a network so as to communicate with each other;

said simulator stations each including a data storage system storing scene data thereon and a computerized image generator generating real-time video as a stream of images each rendered based on the scene data corresponding to a respective point in time, said video being displayed using a display at said simulator station that communicates with said image generator;

said scene data including object data defining positions and physical attributes of virtual objects in the shared simulated environment;

a physics farm associated with the simulator stations, said physics farm interfacing with the scene data of each of the simulator stations and controlling position data of virtual objects in said scene data based on the defined physical attributes of the virtual objects and on physical rules that cause the virtual objects to emulate real movement of objects having corresponding real physical attributes;

said physics farm comprising a plurality of physics engines operatively connected with the network so as to communicate with each other, each of the simulator stations having at least a respective one of the physics engines at its location; and

the physics farm assigning each of the virtual objects in the shared simulated environment to one of the physics engines;

each of the physics engines determining if the virtual objects assigned thereto are in a physical conflict with any other objects in the simulated environment, and, responsive to such a determination, resolving the conflict locally, or if resolution of the conflict locally is not possible, transmitting a packet of data identifying the virtual object involved and including conflict data corresponding to the conflict over the network.

19. The system of claim 18, wherein the physics farm assigns the virtual objects that are relevant to the physics engine of the simulator station, and wherein a virtual object is determined to be relevant to the simulator station based on a distance determined between the associated user location of the simulator station and the location of the virtual object in the simulated environment.

20. The system of claim 18, wherein at least some of the simulator stations include a simulation of a targeted weapon system, and wherein the physics farm assigns the virtual objects that are relevant to the physics engine of the simulator station, and wherein a virtual object is determined to be relevant to a particular one of the work stations based on targeting thereof by the simulation of the weapon system.

21. The system of claim 18, wherein the simulator stations simulate vehicles, and each station has scene data defining virtual objects that make up the respective simulated vehicle, said virtual objects all being assigned to one or more physics engines that are located in the simulator station or that communicate therewith out latency.

22. The system of claim 21, wherein the simulator station simulates the operation of the vehicle using the scene data defining the virtual objects that constitute the simulated vehicle.

23. The system of claim 18, wherein an instructor station communicates with the simulator stations and is able to modify physical attributes of a virtual object.

24. The system of claim 18, wherein a simulation computer system is connected with the network, said simulation computer system deriving client movement data defining movement of a client virtual object in the virtual environment and time-related data defining timing or rate of said movement, and transmitting said client movement data over the network to synchronizers at the simulator stations, said synchronizers receiving said client movement data, and transmitting data to the physics engine of the associated simulator station so as to cause the client virtual object to move in the scene data of the simulator station in compliance with the movement and timing or rate defined in the client movement data.

25. The system of claim 24, wherein the data transmitted to the physics engine is prepared using a closed loop control process using the scene data stored by the data storage system.

26. The system of claim 24, wherein the client movement data is route data defining a route of at least one route point and an ETA or a rate for the client virtual object to move to the point.

27. The system of claim 26, wherein the synchronizer, responsive to detecting that the client virtual object is not moving fast enough to reach the route point by the ETA, or is moving slower than the rate, transmits data to the physics engine that causes the client virtual object to be accelerated in the scene data.

28. The system of claim 25, wherein the synchronizer, responsive to detecting that the client virtual object is obstructed by an object or objects in the virtual environment, transmits data to the physics engine causing the client virtual object to follow a route that circumvents the object or objects.

29. The system of claim 25, wherein the synchronizer, responsive to detecting that the client virtual object is obstructed by an object or objects in the virtual environment, causes the physics engine to at least partially suspend the physics rules thereof so as to permit the client virtual object to move in spite of the obstruction by said object or objects.

30. A system according to claim 18, wherein an SAF application is supported on a computer system communicating with said physics farm, said SAF application transmitting to said physics farm requests for creation or movement of SAF entities in the virtual scene.

31. The system of claim 30 wherein the physics farm transmits to said SAF application data indicative of physics based effects on the SAF entities.

32. The system of claim 30, wherein the SAF application includes a physics manager application and an SAF behavior application, said SAF behavior application transmitting to said physics farm requests based on SAF behavior rules in response to the physics based effects on the SAF entities.

33. A method for providing an interactive simulation of an environment to a plurality of users, said method comprising:

providing to each of said users a respective simulator station having a display displaying to the associated user video rendered by a computerized image generator thereof from scene data stored in a computer accessible data storage system;

receiving from one of said users at the respective simulator station an input signal indicative of an interactive command from the user for influencing the simulated environment;

deriving data from said input signal defining said interactive command as a force applied to one or more virtual objects in the simulated environment;

performing physics processing of the force on the object or objects using a distributed physics farm comprising a plurality of physics engines, at least one of said physics engines being operatively associated with the respective simulator station, said processing publishing physics notification data reporting said force on said object and a system time thereof to all of the physics engines in said physics farm;

receiving the physics notification data at a second of the physics engines operatively associated with another of the simulation stations;

performing physics processing of the force applied to said object at the second of the physics engines, said second physics engine computing a position of the object based on said physics notification data and a difference between the system time thereof and a current system time, said second physics engine accessing locally data defining the positions of other objects in the simulated environment and including in said determination of the computed position a conflict check that incorporates into said computed position any conflict with other objects in the simulated environment;

modifying the scene data at the associated simulation station such that the scene data reflects an updated position of said object conforming to the computed position;

rendering the scene data to produce imagery showing the object at the computed position; and

displaying said imagery to the user of the second simulation station.

34. The method of claim 33 wherein the physics engines all express locations of objects according to a first coordinate system and the scene data at said second simulator system is expressed using a second different coordinate system, said method further comprising converting the computed position of the object to said second different coordinate system for the scene data.

35. The method of claim 33, wherein said physics farm includes a physics manager that selects the specific physics engine that prepares the physics notification data for the object, said selection being made to reduce latency in use of the physics notification data by any affected simulation station.

36. The method of claim 35, wherein said selection is made by the physics manager based on an identification of the simulation station that has a user with a virtual location closest to the object in the simulated environment.

37. The method of claim 35, wherein said selection is made by the physics manager based on an identification of the simulation station that is most closely interacting with the object.

38. The method of claim 33 and further comprising receiving from an instructor station a command directing modification or creation of an object defined in said physics farm, and changing or creating data stored by said physics farm associated with said object in compliance with said instruction.

39. The method of claim 33 wherein, at a start of operation of the simulation, all simulation stations have the same scene data and all physics engines have the dame physics data.

40. The method of claim 33 and further comprising receiving from a second simulator station an input command, deriving therefrom a force applied to a second object in the simulated environment, and processing said force on said second object in the physics farm, transmitting further physics notification data therefor to the physics engine associated with the first simulation station, and displaying at the first simulation station rendered imagery showing said second object in an computed position determined by the associated physics engine.

41. The method of claim 33, wherein corrective position data for the object is received by the physics engine at the second simulation station, and the scene data at the second simulation station is modified to conform to the corrected position data for the object.

42. The method of claim 41, wherein the position is corrected over a series of frames of rendered imagery so as to smooth movement of the object.

43. A simulation system comprising:

a physics farm communications layer comprising a communications backbone;

a physics farm layer having a plurality of distributed physics engines each being communicatively linked to the physics farm communications layer so as to transmit data to the other physics engines and to receive data therefrom, each physics engine storing physical attribute and location data for all of objects in a virtual environment;

a physics farm management system selecting the physics engine that performs physics processing for each object in the virtual environment; and

a physics based simulation API communicating with physics farm management system to allow access thereto by applications; and

a physics based simulation application layer comprising at least two distributed simulation stations providing interactive simulation to respective users including displaying to the respective user imagery rendered from scene data stored at said simulation system, said scene data being modified by said physics farm layer to reflect interaction of the users with the objects in the virtual environment.

44. The system of clam 43, and further comprising a physics based simulation training layer communicating with the physics based applications layer and comprising a training communications infrastructure and an administrator station receiving command inputs for controlling the system.

45. The system of claim 44, wherein the system modifies the physics data responsive to said command inputs from the administrator station.

46. A method of providing distributed simulation of a plurality of vehicles, said method comprising:

providing a plurality of simulation stations each having simulated controls for a respective vehicle being simulated, a display, a storage medium storing scene data defining positions of virtual objects in a virtual environment, and a physics engine storing physics data for objects in the virtual environment, said objects including an ownship set of objects configured to correspond to the vehicles being simulated;

performing, using the physics engine of each simulation station, physics processing of the ownship set of objects for that simulation station and transmitting the resulting physics data to the other simulation station or stations;

receiving the physics data at the other simulation station or stations and modifying the scene data thereof based on said physics data so that said other user or users can see the other station'"'"'s ownship if appropriate to said user'"'"'s point of view; and

using the physics data for each ownship to control the simulated behavior of the ownship of said simulation station.

47. A simulation system comprising:

a plurality of computerized simulation stations connected with each other via a network, each having a respective computer-accessible data storage storing a respective version of scene data corresponding to a virtual environment, a respective computerized image generator generating serial frames each of which is a rendered view for a respective point of view in said virtual environment, and a display device displaying said frames;

a simulation computer system determining behavior of a virtual client entity in the virtual environment and transmitting client movement data via the network to the simulation stations, said client movement data corresponding to said behavior and defining at least one location in the virtual environment to which the client entity is to move, and time-constraint data indicative of a point in time by which the entity is to move to the location; and

each simulation station having a synchronizer receiving the client movement data and based thereon causing the scene data to be modified so as to reflect movement of the client entity to said location by said point in time.

48. The system of claim 47, wherein the client movement data identifies the location as a point in the virtual environment.

49. The system of claim 48, wherein, where the client entity is a ground-moving entity, the client movement data identifies the location by a two-dimensional coordinate in a two-dimensional surface in the virtual environment in which the client entity moves.

50. The system of claim 48, wherein, where the client entity is an air-moving entity, the client movement data identifies the location by a three-dimensional coordinate in a three-dimensional space in the virtual environment in which the client entity moves.

51. The system of claim 47, wherein the client movement data identifies an orientation in the virtual environment for the client entity.

52. The system of claim 47, wherein the client movement data comprises data defining a point in time at which the entity is directed to reach the location.

53. The system of claim 47, wherein the client movement data comprises data defining a rate at which the entity is directed to move toward the location.

54. The system of claim 53, wherein the client movement data comprises data defining a start time at which the entity is directed to start moving toward the location.

55. The system of claim 47, wherein the synchronizer controls the movement of the client entity in the scene data using a closed loop control process.

56. The system of claim 47, wherein each simulation station has a physics engine accessing physics based environment data stored at said simulation station corresponding to all virtual objects in the virtual environment of the scene data, said physics engine controlling the scene data such that the virtual objects in the scene data move according to the physics based environment data and physics rules implemented by said physics engine.

57. The system of claim 56, wherein each synchronizer influences the respective version of the scene data by transmitting data to the physics engine configured to produce movement of the client entity in the virtual environment.

58. The system of claim 56, wherein positional update data is transmitted to the simulation stations so that the scene data is updated continually so as to be in conformance with all other versions of the scene data across the system.

59. The system of claim 56, wherein the client entity is directed to move along a route to said location, and wherein, if the client entity is prevented from movement toward the location by the physics engine due to obstruction of said route by an object or objects in the virtual environment, a new route to the location is devised that avoids being obstructed by said object or objects, and said client entity is caused by said synchronizer to move along said new route so as to reach the location by said point in time.

60. The system of claim 56, wherein the client entity is directed to move along a route to said location, and wherein, if the client entity is prevented from movement toward the location by the physics engine due to obstruction of said route by an object or objects in the virtual environment, said synchronizer causes the physics engine to temporarily suspend at least part of the physics rules thereof with respect to the object or objects such that the client entity is able to move without being obstructed thereby and so as to reach the location by said point in time.

61. The system of claim 56, wherein, responsive to the client movement data, the synchronizer is directed to move along a first route to said location, and, if the client entity is prevented from movement toward the location by the physics engine due to obstruction of said route by an object or objects in the virtual environment, the synchronizer attempts to devise a new route to the location that avoids being obstructed by said object or objects, wherein, if the synchronizer can devise the new route, said client entity is caused by said synchronizer to move along said new route so as to reach the location by said point in time;

and

wherein, if the synchronizer can devise the new route, said synchronizer causes the physics engine to temporarily suspend at least part of the physics rules thereof with respect to the object or objects such that the client entity is able to move along the route without being obstructed thereby and so as to reach the location by said point in time.

62. The system of claim 56, wherein the synchronizer derives from the client movement data a desired route for the client entity to follow to the location and a desired rate at which to proceed along said route, the synchronizer using a closed loop speed control process to control the rate of movement of the client entity along said route based on the desired rate, and the synchronizer using a closed loop steering control process to control the direction of movement of the client entity so that said client entity follows said route.

63. A method for providing an interactive simulation of a virtual environment to a user at a simulator station, said method comprising:

storing physics data on a computer accessible memory coupled with a physics engine, said physics data defining physics parameters of a plurality of virtual objects in a virtual scene, and including position data reflecting respective positions of the virtual objects in the virtual scene;

passing to the physics engine data identifying at least one of the virtual objects and defining a value of force or torque, said physics engine modifying the physics data in the memory so that the position data of the virtual object in the memory reflects a modified position that corresponds to a result in the virtual environment of the force or torque acting on the virtual object pursuant to physics rules of said physics engine;

obtaining and storing said modified position data so as to be accessible to a computerized viewer; and

rendering an image corresponding to a view of the virtual scene using said computerized viewer, said view including a view of at least a portion of said virtual object in said modified position.

64. The method of claim 63 wherein the method further comprises receiving route data at said simulator station over a network connected therewith, said route data defining for the virtual object a point in the scene and a time by which said virtual object is to move to said point in the scene;

and

determining a first force or torque value to be applied to the virtual object or a part thereof configured to move said virtual object to the point in the scene by the time; and

passing said first force or torque value to said physics engine; and

obtaining first position data of the virtual object after processing of said first force or torque value by the physics engine.

65. The method of claim 64 wherein the method further comprises determining a second force or torque value to be applied to the virtual object or a part thereof configured to move said virtual object to the point in the scene by the time based on the first position data for the virtual object.

66. The method of claim 65 wherein the method further comprises storing said first position data of the virtual object;

and

rendering a second image corresponding to a view of the virtual scene using said computerized viewer, said view including a view of at least a portion of said virtual object in the position defined by said first position data.

67. The method of claim 65 wherein the method further comprises determining whether the virtual object is delayed in reaching the point in the scene by being obstructed by another virtual object defined by the physics data in the virtual scene;

and

taking action responsive to a determination of obstruction to reach the point by the time;

said action comprising determining a route to the point that avoids the obstructing virtual object and determining a corrective force or torque value configured to cause said virtual object to follow said route, and passing said corrective force or torque to said physics engine.

68. The method of claim 65 wherein the method further comprises determining whether the distance in the virtual scene between the first position and the previous position is less than the distance to be covered by the virtual object in the virtual scene, and responsive to such a comparison, increasing the force or torque value for the virtual object so as to cause its acceleration.

69. The method of claim 65 wherein the method further comprises determining whether the virtual object is delayed in reaching the point in the scene by being obstructed by another virtual object defined by the physics data in the virtual scene;

and

taking action responsive to a determination of obstruction to reach the point by the time;

said action comprising causing said physics engine to modify the physics rules thereof so that said obstructing virtual object does not obstruct the virtual object in moving to the point.

70. The method of claim 65 wherein the view is displayed to the user as a single frame in a series of frames that form a video of the scene shown to the user, and said rendering steps are separated by a period of time that is no longer than a cycle of display of sequential images in the video.

71. A system for providing distributed simulations to one or more users, said system comprising:

a first computer system linked for communication over a network and having a data storage device storing data and a display;

said data storage device storing data defining a virtual scene having a number of virtual objects therein each having a respective visual appearance, position and orientation defined by said data;

said computer system including a viewer generating real time imagery from said data and displaying said imagery on said display, the imagery corresponding to a rendered view at the time the image was rendered of the virtual scene as defined by said data;

a second computer system having a data storage device, said data storage device storing data defining the same virtual scene with the same virtual objects as defined by said data on the first computer system;

said second computer system running a program determining movement of a controlled virtual object in the virtual scene;

said second computer system further having a master synchronizer transmitting over the network route data for said controlled virtual object indicative of a point in the virtual scene and a future point in time by which the controlled virtual object is directed to reach said point;

said first computer system having a slave synchronizer receiving said route data, and responsive thereto, causing the data stored therein to be modified so that in the data stored at the first computer system, the controlled virtual object moves to the point in the virtual scene by the point in time.

72. The system of claim 71 wherein the point in time is defined in terms of a shared system clock value.

73. The system of claim 71 wherein the route data comprises data defining a point and data defining a point in time.

74. The system of claim 71 wherein the route data comprises data defining a point in the scene, a start time and a rate.

75. The system of claim 71 wherein the first computer system has a master synchronizer that begins to transmit route data for the virtual object to the second computer system, and said second computer system has a slave synchronizer that modifies the data stored at the second computer system so as to comply with said route data, and the master synchronizer at the second computer system relinquishes control of the controlled virtual object and the slave synchronizer at the first computer system does not cause further changes in the data at the first computer system.

76. The system of claim 71 wherein each computer system has a physics engine with memory therein storing physics data corresponding to physical parameters of the virtual objects in the virtual scene including respective positions thereof, said scene data at each computer system being periodically updated by position data from the physics engine thereof.

77. The system of claim 76 wherein the slave synchronizer at the first computer system determines whether the virtual object is delayed in proceeding to said point by said time, and responsive to said determination, takes a corrective action selected from the group consisting of increasing force on the virtual object, causing the virtual object to move on a route around an obstruction, and modifying or suspending physical rules of the physics engine so as to permit the virtual object to reach the point by the time.