Visualizing space systems modeling using augmented reality

• US 7,693,702 B1
• Filed: 10/31/2003
• Issued: 04/06/2010
• Est. Priority Date: 11/01/2002
• Status: Active Grant

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Stiles, Randy J., Kerner, Sara A.
• Primary Examiner(s): Shah; Kamini S
• Assistant Examiner(s): PATEL, SHAMBHAVI K
• Application Number:
  US10/699,576
• Time in Patent Office: 2,349 Days
• Field of Search: 703/21, 703/22, 345/7, 345/8, 345/9, 345/629, 701/13, 715/700, 348/77
• US Class Current: 703/22
• CPC Class Codes: B64G 1/242 Orbits and trajectories
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A visualization system for a computer system includes a modeling portion configured to specify the virtual geographic location in response to a three-dimensional model of the virtual geographic location and in response to the position of the viewer with respect to the virtual geographic location, a satellite data portion configured to specify a representation of satellite status data in response to the position of the viewer with respect to the virtual geographic location and in response to the satellite status data, and a three-dimensional output portion configured to provide at least two images of the virtual geographic location and the representation of the satellite status data to the viewer.

Claims

1. A visualization system for augmented reality, the visualization system for developing a three-dimensional representation of a space system, the visualization system comprising:

a tangible medium comprising;

a positioning portion configured to determine a position of a viewer with respect to a real world and a position of the viewer with respect to a virtual world, the positioning portion configured to allow the viewer to interact with the virtual world;

a modeling portion configured to specify the virtual world in response to a space system model of the virtual world, the space system model including models for the earth and satellites, the space system model based on historical and projected trajectories of satellites;

a model specification portion configured to specify a representation of satellite model status data of the satellites in response to the position of the viewer with respect to the virtual world and in response to satellite model status data of the satellites;

an output portion configured to provide a three-dimensional representation of a space system, the three-dimensional representation of the space system including an image of the virtual world super-imposed on an image of the real world as being seen by the viewer, the image of the virtual world including the representation of the satellite model status data of the satellites to the viewer in response to the position of the viewer with respect to the virtual world, the representation of the satellite model status data of the satellites including a three-dimensional representation of satellite orbits, the image of the virtual world including a three-dimensional representation of the models for the earth and the satellites;

a simulation portion configured to select a virtual world time from a past time, a future time, a slower than real time, a static time and a faster than real time, run a simulation of the virtual world in the selected virtual world time, extract a position of an object in the space system model based on the virtual world time and update the image of the virtual world based on the extracted position; and

an input portion configured to allow the viewer to select one of the satellites to view satellite model status data of the selected one of the satellites and configured to allow the viewer to move the selected one of the satellites to a different position,

wherein the representation of the satellite model status data of the satellites further comprises a representation selected from the group;

satellite sensor orientation, satellite sensor position, and satellite system design data.

2. The visualization system of claim 1 further comprising:

an image acquisition source configured to capture at least an image comprising an image of the real world, and an image of at least a pre-determined marker positioned in the real world,

wherein the positioning portion comprises;

an image processing portion configured to determine the position of the viewer with respect to the real world in response to the image of the pre-determined marker; and

a virtual positioning portion configured to translate the position of the viewer in the real world to the position of the viewer in the virtual world.

3. The visualization system of claim 1 wherein a model of the virtual world is a multi-dimensional model of the virtual world, and the output portion is a multi-dimensional output portion.

4. The visualization system of claim 1 wherein the representation of the satellite model status data further comprises a representation selected from the group:

a current position of a satellite, a past position of a satellite, a future position of a satellite, an orientation of a satellite, a trajectory of a satellite, ground coverage of a satellite, a satellite'"'"'s orientation vectors to other satellites or objects, a satellite'"'"'s coverage analysis when the satellite is in a view of a region, satellite revisit time, a satellite communication link or network, beam strength of space, and satellite systems status.

5. The visualization system of claim 1 further comprising a heads-up pair of glasses.

6. The visualization system of claim 1 wherein the visualization system is configured to allow more than one user to interact with the image of the virtual world at the same time and is configured to allow the more than one user to collaborate with each other while viewing from different positions.

7. The visualization system of claim 5 wherein the heads-up pair of glasses are also configured to allow the viewer to view the image of the virtual world super-imposed on the image of the real world.

21. The visualization system of claim 1 wherein the image of the real world and the image of the virtual world are provided in real-time.

24. The visualization system of claim 1 wherein the viewer is allowed to perform one or more of the following:

selecting a satellite, selecting an orbit, selecting a geographic area, or zooming-in or out of the virtual world.

25. The visualization system of claim 1 wherein the viewer is allowed to directly select and manipulate objects in the virtual world without using a mouse.

26. The visualization system of claim 1 wherein the positioning portion comprises a marker positioned in the real world, the marker is static or is placed upon a paddle that includes a pre-defined visual marker in the real world, and the paddle is capable of being moved around the real world.

8. A method for visualization of augmented reality to develop a three-dimensional representation of a space system, the method comprising:

determining a position of a viewer with respect to a real world and a position of the viewer with respect to a virtual world;

determining a space system model of the virtual world, the space system model including models for the earth and satellites, the space system model based on historical and projected trajectories of satellites;

determining a representation of satellite model status data of the satellites in response to the position of the viewer with respect to the virtual world and in response to satellite model status data of the satellites;

displaying to the viewer a three-dimensional representation of a space system, the three-dimensional representation of the space system including a representation of the virtual world super-imposed on a representation of the real world as being seen by the viewer, the representation of the virtual world including the representation of the satellite model status data of the satellites in response to the position of the viewer with respect to the virtual world, the representation of the satellite model status data of the satellites including a three-dimensional representation of satellite orbits, the image of the virtual world including a three-dimensional representation of the models for the earth and the satellites;

selecting one of the satellites, by the viewer, to view satellite model status data of the selected one of the satellites; and

moving the selected one of the satellites, by the viewer, to a different position,

selecting a virtual world time from a past time, a future time, a slower than real time, a static time and a faster than real time;

running a simulation of the virtual world in the selected virtual world time, and

updating the image of the virtual world based on the virtual world time by extracting a position of an object in the space system model;

wherein the representation of the satellite model status data of the satellites further comprises a representation selected from the group;

satellite sensor orientation, satellite sensor position, and satellite system design data.

9. The method of claim 8 wherein the step of determining the position of the viewer comprises:

capturing of at least one pre-determined marker positioned in the real world;

determining a position and orientation of the viewer with respect to the real world in response to the image of the pre-determined marker; and

determining the position of the viewer with respect to the virtual world in response to determining the position and orientation of the viewer with respect to the real world.

10. The method of claim 8 wherein a model of the virtual world is a multi-dimensional model of the virtual world, and the representation of the virtual world is a multi-dimensional representation of the virtual world.

11. The method of claim 8 wherein the representation of the satellite model status data further comprises beam strength of land based laser devices, and the image of the real world as being seen by the viewer is an image of a real world stage as being seen by the viewer.

12. The method of claim 8 wherein the step of displaying comprises displaying the representation of the real world and the representation of the virtual world to the viewer with a pair of heads-up glasses.

13. The method of claim 9 wherein the step of capturing the image of the pre-determined marker uses a video camera, and the video camera is disposed upon the pair of head-up glasses.

14. The method of claim 8 wherein the viewer views the representation of the real world at the same time as the representation of the virtual world.

22. The method of claim 8 wherein the step of displaying comprises displaying to the viewer in real time the representation of the real world and the representation of the virtual world.

15. A method for visualization of augmented reality, the method for developing a three-dimensional representation of a space system, the method comprising:

determining a space system model of a virtual world, the space system model including models for the earth and satellites, the space system model based on historical and projected trajectories of satellites;

determining a representation of satellite model status data of the satellites in response to satellite model status data of the satellites, and in response to a position of a viewer with respect to the virtual world;

determining a representation of the virtual world in response to the space system model of the virtual world and in response to a position of the viewer with respect to the virtual world;

displaying to the viewer a three-dimensional representation of a space system, the three-dimensional representation of the space system including a representation of a real world as being seen by the viewer overlaid with the representation of the virtual world, the representation of the virtual world including the representation of the satellite model status data of the satellites, the representation of the satellite model status data of the satellites including a three-dimensional representation of satellite orbits, the representation of the virtual world including a three-dimensional representation of the models for the earth and the satellites;

selecting one of the satellites, by the viewer, to view satellite model status data of the selected one of the satellites;

moving the selected one of the satellites, by the viewer, to a different position,

selecting a virtual world time from a past time, a future time, a slower than real time, a static time and a faster than real time;

running a simulation of the virtual world in the selected virtual world time; and

updating the image of the virtual world based on the virtual world time by extracting a position of an object in the space system model;

wherein the viewer is allowed to interact with the virtual world.

16. The visualization method of claim 15 wherein the position of the viewer with respect to the virtual world is determined in response to an image of a pre-determined marker positioned in the real world taken from a vantage point of the viewer and in response to a correspondence between the virtual world and the real world.

17. The visualization method of claim 15 wherein the representation of the satellite model status data of the satellites comprises missile threat clouds, and the image of the real world as being seen by the viewer is an image of a real world stage as being seen by the viewer.

18. The visualization method of claim 15 wherein the representation of the virtual world and the representation of the real world are provided to the viewer with a pair of heads-up display glasses.

19. The visualization method of claim 15 further comprising displaying to the viewer a portion of the virtual world selected by the viewer,

wherein the viewer selection is determined in response to a position of a viewer-controlled marker with respect to the virtual world, wherein the marker is positioned in the real world.

20. The visualization method of claim 19 wherein the step of displaying to the viewer the portion of the virtual world selected by the viewer comprises overlaying an icon over the portion of the virtual world displayed to the viewer.

23. The visualization method of claim 15 wherein the step of displaying comprises displaying to the viewer in real time the representation of the real world overlaid with the representation of the virtual world.

27. A visualization system for augmented reality, the visualization system for developing a three-dimensional representation of a space system, the visualization system comprising:

a processor for executing instructions, the instructions comprising;

determining a position of a viewer with respect a real world and a position of the viewer with respect to a virtual world;

specifying the virtual world in response to a space system model of the virtual world, the space system model including models for the earth and satellites, the space system model based on historical and projected trajectories of satellites;

specifying a representation of satellite model status data of the satellites in response to the position of the viewer with respect to the virtual world and in response to satellite model status data of the satellites;

providing a three-dimensional representation of a space system, the three-dimensional representation of the space system including an image of the virtual world super-imposed on an image of the real world as being seen by the viewer, the image of the virtual world including the representation of the satellite model status data of the satellites to the viewer in response to the position of the viewer with respect to the virtual world, the representation of the satellite model status data of the satellites including a three-dimensional representation of satellite orbits, the image of the virtual world including a three-dimensional representation of the models for the earth and the satellites;

selecting a virtual world time from a past time, a future time, a slower than real time, a static time and a faster than real time;

running a simulation of the virtual world in the selected virtual world time;

updating the image of the virtual world based on the virtual world time by extracting a position of an object in the space system model; and

allowing the viewer to select one of the satellites to view satellite model status data of the selected one of the satellites and allowing the viewer to move the selected one of the satellites to a different position.

28. The visualization system of claim 27 wherein the representation of the satellite model status data of the satellites comprises a representation selected from the group:

a current position of a satellite, a past position of a satellite, a future position of a satellite, an orientation of a satellite, ground coverage of a satellite, a trajectory of a satellite, satellite sensor orientation, satellite sensor position, a satellite'"'"'s orientation vectors to other satellites or objects, a satellite'"'"'s coverage analysis when the satellite is in a view of a region, satellite revisit time, a satellite communication link or network, missile threat clouds, beam strength of space and land based laser devices, satellite systems status, and satellite system design data.