System and Method for Creating and Displaying Map Projections related to Real-Time Images

• US 20110188760A1
• Filed: 02/03/2010
• Published: 08/04/2011
• Est. Priority Date: 02/03/2010
• Status: Active Grant

• Current Assignee: Uncharted Software, Inc.
• Original Assignee: Uncharted Software, Inc.
• Inventors: SENIOR, Robin, KAPLER, Thomas, WRIGHT, William
• Granted Patent: US 8,436,872 B2
• Time in Patent Office: Days
• US Class Current: 382/203
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09B 29/106 using electronic means

Abstract

There is provided a method and system for creating and displaying a map projection of a device'"'"'s real-time viewing area to depict virtual objects, the virtual objects providing a reflected view of real-time objects displayed within the device'"'"'s viewing area, the method comprising: displaying a real-time image of the device'"'"'s viewing area taken from a geographical location on a display; retrieving the map projection for revealing the reflected view as an elevated view of a ground surface about the device'"'"'s current geographical location and in accordance with the device'"'"'s viewing area; superimposing the map projection on the display and overlaid in an upper portion of the real-time image; and defining one or more markers configured to show a relationship between the map projection and the real-time image, each marker overlaid on the display and configured to connect between the virtual object in the map projection and the corresponding real-time object on the real-time image.

Claims

1. A method for creating and displaying a map projection of a device'"'"'s real-time viewing area to depict virtual objects, the virtual objects providing a reflected view of real-time objects displayed within the device'"'"'s viewing area, the method comprising:

displaying a real-time image of the device'"'"'s viewing area taken from a geographical location on a display;

retrieving the map projection for revealing the reflected view as an elevated view of a ground surface about the device'"'"'s current geographical location and in accordance with the device'"'"'s viewing area;

superimposing the map projection on the display and overlaid in an upper portion of the real-time image; and

defining one or more markers configured to show a relationship between the map projection and the real-time image, each marker overlaid on the display and configured to connect between the virtual object in the map projection and the corresponding real-time object on the real-time image.

2. A method according to claim 1, further comprising:

providing a user interface configured to interact with a user of the device for allowing one or more of selecting virtual objects, zooming, panning, drilling down on the map projection, changing shape of the map projection surface, adjusting transparency of the map projection, and adjusting height of the map projection surface.

3. A method according to claim 1, further comprising:

applying a transformation to the map projection to obtain a transformed map projection having a curved surface, the curved surface having at least one portion with a convex shape for magnifying virtual objects within the map projection that are located close to the device and compressing virtual objects located away from the device.

4. A method according to claim 1, wherein the map projection is a flat image positioned directly above the real-time image to provide a vertical correspondence between each virtual object and corresponding real-time object.

5. A method according to claim 4, further comprising the step of transforming the map projection by shifting the map projection and scaling thereof relative to the real-time image to provide a transformed image, the transformed image configured to provide magnification of at least a portion of the real-time image.

6. A method according to claim 5, further comprising the step of applying a slope to the transformed image to obtain a tilted transformed image, the tilted transformed image configured to depict the virtual objects at an angle relative to the corresponding real-time objects.

7. A method according to claim 1, wherein the map projection is transformed by applying a multiple axis curve to the map projection, the transformed image configured to provide magnification of at least a portion of the real-time image located closest from the device'"'"'s current geographical location.

8. A method according to claim 1, wherein the map projection is transformed by applying a curved surface transform to the map projection, the curvature of the curved surface being adaptive in response to a change in a viewing angle of the device.

9. A method according to claim 1, wherein the map projection is transformed by scaling the map projection along two separate axes in dependence upon a distance of the virtual object in the map projection to the geographical location of the device.

10. A method according to claim 1, wherein the map projection is transformed by magnifying virtual objects corresponding to real-time objects located at a pre-defined distance from the device'"'"'s geographical location.

11. A method according to claim 1, wherein the map projection displays virtual objects corresponding to a subset of real-time objects within the device'"'"'s viewing area to provide a partial reflection of the real-time image.

12. A method according to claim 1, wherein the one or more markers are defined by colour coding a selected one of the virtual objects to correspond to a colour of a corresponding one of the real-time objects.

13. A method according to claim 1, wherein the map projection is a vector image providing a symbolic reflection of real-time objects as corresponding virtual objects.

14. A method according to claim 2, wherein the user interface is further configured to interact with a user to allow placement and modification of at least one of the one or more markers, annotations, vectors, and symbols each configured for defining a relationship between the map projection and the real-time image.

15. A method according to claim 1, wherein the map projection is one of semi-transparent or transparent image to allow visibility of the upper portion of the real-time image.

16. A method according to claim 1, further comprising the step of determining the upper portion of the real-time image for overlaying the map projection thereon, wherein determining the upper portion comprises determining a portion of the real-time image having a least number of real-time objects.

17. A method according to claim 1, further comprising the step of receiving physical characteristic information of at least one of the real-time objects, wherein each marker extends between the virtual object and a top surface of the corresponding real-time object in accordance with the physical characteristic information of the corresponding real-time object.

18. A system for creating and displaying a map projection of a device'"'"'s real-time viewing area to depict virtual objects, the virtual objects providing a reflected view of real-time objects displayed within the device'"'"'s viewing area, the system comprising:

a visualization renderer for displaying a real-time image of the device'"'"'s viewing area taken from a geographical location on a display;

an extraction module for retrieving the map projection for revealing the reflected view as an elevated view of a ground surface about the device'"'"'s current geographical location and in accordance with the device'"'"'s viewing area, the extraction module coupled to the visualization renderer for superimposing the map projection on the display and overlaid in an upper portion of the real-time image;

an associations module coupled to the extraction module for defining one or more markers configured to show a relationship between the map projection and the real-time image; and

a leader module coupled to the associations module for connecting each marker between the virtual object in the map projection and the corresponding real-time object on the real-time image, the leader module communicating with the visualization renderer for overlaying each marker on the display on top of the real-time image and the map projection.

19. The system according to claim 18 further comprising a transform module coupled to the associations module, the transform module configured to apply a transformation to the map projection to obtain a transformed map projection having a curved surface, the curved surface having at least one portion with a convex shape for magnifying virtual objects within the map projection that are located close to the device and compressing virtual objects located away from the device.

20. The system according to claim 18 further comprising a transform module coupled to the associations module, the transform module configured to apply a slope to the projection map to obtain a tilted transformed image, the tilted transformed image configured to depict the virtual objects at an angle relative to the corresponding real-time objects.

21. The system according to claim 18 further comprising a transform module coupled to the associations module, the transform module configured to apply a multiple axis curve to the map projection, the transformed image configured to provide magnification of at least a portion of the real-time image located closest from the device'"'"'s current geographical location.