System and method for creating and displaying map projections related to real-time images

• US 8,436,872 B2
• Filed: 02/03/2010
• Issued: 05/07/2013
• Est. Priority Date: 02/03/2010
• Status: Active Grant

• Current Assignee: Uncharted Software, Inc.
• Original Assignee: Uncharted Software, Inc.
• Inventors: Wright, William, Kapler, Thomas, Senior, Robin
• Primary Examiner(s): Thompson, James A
• Assistant Examiner(s): Chu, David H
• Application Number:
  US12/699,545
• Publication Number:
  US 20110188760A1
• Time in Patent Office: 1,189 Days
• Field of Search: None
• US Class Current: 345/633
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09B 29/106 using electronic means

Abstract

There is provided a method and system for creating and displaying a map projection of a device'"'"'s real-time viewing area to depict virtual objects, the virtual objects providing a reflected view of real-time objects displayed within the device'"'"'s viewing area, the method comprising: displaying a real-time image of the device'"'"'s viewing area taken from a geographical location on a display; retrieving the map projection for revealing the reflected view as an elevated view of a ground surface about the device'"'"'s current geographical location and in accordance with the device'"'"'s viewing area; superimposing the map projection on the display and overlaid in an upper portion of the real-time image; and defining one or more markers configured to show a relationship between the map projection and the real-time image, each marker overlaid on the display and configured to connect between the virtual object in the map projection and the corresponding real-time object on the real-time image.

Claims

1. A method for creating and displaying a transformation of a map projection of a device'"'"'s real-time viewing area to depict virtual objects, the virtual objects providing a reflected view of real-time objects displayed within the device'"'"'s viewing area, the method comprising:

displaying a real-time image of the device'"'"'s viewing area taken from a geographical location on a display screen of the device;

determining the map projection for revealing, the reflected view as an elevated view of a ground surface about the device'"'"'s current geographical location and in accordance with the device'"'"'s viewing area;

transforming the map projection by applying one of;

a curved surface transform to the map projection such that the curvature of the curved surface is adaptive to a change in a viewing angle of the device and scaling the map projection along two separate axes in dependence upon a distance of the virtual object in the map projection to the geographical location of the device;

superimposing the transformed map projection on the display screen and overlaid in an upper portion of the real-time image; and

defining one or more markers configured to show a relationship between the transformed map projection and the real-time image, each marker overlaid on the display and configured to connect between the virtual object in the transformed map projection and the corresponding real-time object on the real-time image.

2. A method according to claim 1, further comprising:

providing a user interface configured to interact with a user of the device for allowing one or more of selecting virtual objects, zooming, panning, drilling down on the transformed map projection, changing shape of the transformed map projection surface, adjusting transparency of the transformed map projection, and adjusting height of the transformed map projection surface.

3. A method according to claim 1, further comprising:

if the map projection is transformed by applying a curved surface, the curved surface includes at least one portion with a convex shape for magnifying virtual objects within the map projection that are located close to the device and compressing virtual objects located away from the device.

4. A method according to claim 1, wherein the transformed map projection is positioned directly above the real-time image to provide a vertical correspondence between each virtual object and corresponding real-time object.

5. A method according to claim 4, further transforming the map projection by shifting the map projection and scaling thereof relative to the real-time image to provide a transformed image, the transformed image configured to provide magnification of at least a portion of the real-time image.

6. A method according to claim 1, further comprising the step of applying a slope to the transformed map projection to obtain a tilted transformed image, the tilted transformed image configured to depict the virtual objects at an angle relative to the corresponding real-time objects.

7. A method according to claim 1, wherein the transformed map projection is further transformed by magnifying virtual objects corresponding to real-time objects located at a pre-defined distance from the device'"'"'s geographical location.

8. A method according to claim 1, wherein the transformed map projection displays virtual objects corresponding to a subset of real-time objects within the device'"'"'s viewing area to provide a partial reflection of the real-time image.

9. A method according to claim 1, wherein the one or more markers are defined by colour coding a selected one of the virtual objects to correspond to a colour of a corresponding one of the real-time objects.

10. A method according to claim 1, wherein the transformed map projection is a vector image providing a symbolic reflection of real-time objects as corresponding virtual objects.

11. A method according to claim 2, wherein the user interface is further configured to interact with a user to allow placement and modification of at least one of the one or more markers, annotations, vectors, and symbols each configured for defining a relationship between the transformed map projection and the real-time image.

12. A method according to claim 1, wherein the transformed map projection being superimposed on the display screen is one of semi-transparent or transparent image to allow visibility of the upper portion of the real-time image.

13. A method according to claim 1, further comprising the step of determining the upper portion of the real-time image for overlaying the transformed map projection thereon, wherein determining the upper portion comprises determining a portion of the real-time image having a least number of real-time objects.

14. A method according to claim 1, further comprising the step of receiving physical characteristic information of at least one of the real-time objects, wherein each marker extends between the virtual object and a top surface of the corresponding real-time object in accordance with the physical characteristic information of the corresponding real-time object.

15. The method of claim 1, wherein the markers are configured to show a visual relationship linking the displayed transformed map projection and the real-time image such that the displayed transformed map projection and markers are configured for assisting in visual navigation.

16. The method of claim 1, wherein determining the map projection is based upon retrieving satellite and/or aerial imagery information for use in determining the reflected view.

17. The method of claim 1, wherein the real-time image and at least one of the transformed map projection, and the markers are updated in response to user events related to the device comprising at least one of a change of location, positioning and angle of the device'"'"'s viewing screen.

18. A system for creating and displaying a transformed map projection of a device'"'"'s real-time viewing area to depict virtual objects, the virtual objects providing a reflected view of real-time objects displayed within the device'"'"'s viewing area, the system comprising:

a visualization renderer for displaying a real-time image of the device'"'"'s viewing area taken from a geographical location on a display screen of the device;

an extraction module for retrieving the map projection for revealing the reflected view as an elevated view of a ground surface about the device'"'"'s current geographical location and in accordance with the device'"'"'s viewing area, the extraction module coupled to the visualization renderer for superimposing the map projection on the display screen and overlaid in an upper portion of the real-time image;

an associations module coupled to the extraction module for defining one or more markers configured to show a relationship between the map projection and the real-time image;

a transform module coupled to the associations module, the transform module configured to apply a transformation to the map projection to obtain a transformed map projection by applying one of;

a curved surface transform to the map projection such that the curvature of the curved surface is adaptive to a change in a viewing angle of the device and scaling the map projection along two separate axes in dependence upon a distance of the virtual object in the map projection to the geographical location of the device; and

a leader module coupled to the associations module and the transform module for connecting each marker between the virtual object in the transformed map projection and the corresponding real-time object on the real-time image, the leader module communicating with the visualization renderer for overlaying each marker on the display screen on top of the real-time image and the transformed map projection.

19. The system according to claim 18, wherein if the map projection is transformed by applying a curved surface, the curved surface includes at least one portion with a convex shape for magnifying virtual objects within the map projection that are located close to the device and compressing virtual objects located away from the device.

20. The system according to claim 18 wherein the transform module is further configured to apply a slope to the transformed map projection to obtain a tilted transformed image, the tilted transformed image configured to depict the virtual objects at an angle relative to the corresponding real-time objects.