Image extraction and image-based rendering for manifolds of terrestrial and aerial visualizations

• US 9,786,075 B2
• Filed: 03/14/2014
• Issued: 10/10/2017
• Est. Priority Date: 06/07/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Aguera-Arcas, Blaise Hilary
• Primary Examiner(s): Johnson, M Good
• Application Number:
  US14/214,343
• Publication Number:
  US 20140362108A1
• Time in Patent Office: 1,306 Days
• Field of Search: None
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/005 Tree description, e.g. octree, quadtree
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation

Abstract

Architecture that extracts (analyzes and selects) a set of images from repositories or sources imagery to create one-dimensional (1D) and two-dimensional (2D) image manifolds (or image packets—also referred to as “synth” packets). The repositories of map imagery can include aerial views and terrestrial views (from points on the earth surface). The manifolds facilitate fluid translation between manifold images thereby providing a fluid user interactive navigation experience for viewing the location.

Claims

1. A computing device, comprising:

one or more processors coupled to a computer-readable storage device, the computer-readable storage device having stored thereon a plurality of instructions causing the one or more processors to;

extract images related to a three-dimensional real world location, from image sources that include at least one of structured aerial-based image sources or structured ground-based image sources based on the location;

format the extracted images from the image sources into a suitable format for a two-dimensional visualization template; and

arrange the extracted images according to the two-dimensional visualization template, the two-dimensional visualization template comprising a local graph that specifies relationship information between the extracted images, wherein the two-dimensional visualization template is configured to enable translation between the extracted images via a viewer; and

create a graph-oriented map that enables linking and navigation between the two-dimensional visualization template and other visualization templates to provide, on a display associated with a computing device, an interactive multi-dimensional visualization for a continuously navigable experience of the real world location.

2. The computing device of claim 1, wherein the two-dimensional visualization template or at least one of the other visualization templates represents a street-level view of the location based on street-level shots extracted from the structured ground-based image sources.

3. The computing device of claim 1, wherein the two-dimensional visualization template or at least one of the other visualization templates represents an aerial view of the location based on aerial shots extracted from the structured aerial-based image sources.

4. The computing device of claim 1, wherein the plurality of instructions further cause the one or more processors to extract perspective images of forward shots and side shots from a panorama image.

5. The computing device of claim 1, wherein the two-dimensional visualization template or at least one of the other visualization templates enables the perception of image changes in time and space via the viewer.

6. The computing device of claim 1, wherein the two-dimensional visualization template or at least one of the other visualization templates enables the play of time information that corresponds to navigation through the images of the two-dimensional visualization template or the at least one of the other visualization templates.

7. The computing device of claim 1, wherein the plurality of instructions further cause the one or more processors to create a graph of spatially-related packets of templates obtained from a map and structured hierarchically for interactive transitions between the packets.

8. A method, comprising acts of:

selecting a two-dimensional visualization template by which to render a real world location for interactive navigation;

extracting images related to the real-world location from three-dimensional image collections for use with the two-dimensional visualization template, the two-dimensional visualization template comprising a local graph that specifies relationship information between the extracted images;

arranging the images according to the two-dimensional visualization template to enable translation between the images; and

creating a graph-oriented map that enables linking and navigation between the two-dimensional visualization template and other visualization templates to provide, on a display associated with a computing device, an interactive multi-dimensional visualization for a continuously navigable experience of the real world location via the images.

9. The method of claim 8, further comprising creating photorealistic transitions between multiple content types of the images according to the two-dimensional visualization template.

10. The method of claim 8, further comprising formatting image content to be compatible with the two-dimensional visualization template.

11. The method of claim 8, further comprising formatting image content from sources that comprise structured aerial images, structured ground-based images, and crowd-sourced images.

12. The method of claim 8, further comprising rendering the images of the two-dimensional visualization template in correspondence to time of day.

13. The method of claim 8, further comprising enabling rendering of the two-dimensional visualization template as one of a panorama, wall, walk, or a spin.

14. A computer-readable hardware storage device comprising computer-executable instructions that when executed by a microprocessor, cause the microprocessor to perform acts of:

extracting images from three-dimensional collections of images;

formatting the extracted images for utilization in a plurality of visualization templates that include one-dimensional templates and two-dimensional templates, wherein at least one of the two-dimensional templates comprises a local graph that specifies relationship information between the extracted images;

arranging the images according to the plurality of visualization templates to enable translation between the images of a visualization template in response to an interaction;

creating a graph-oriented map that enables linking and navigation between the plurality of visualization templates; and

enabling, on a display associated with a computing device, an interactive multi-dimensional visualization for a continuously navigable experience of a real world location based on rendering of a visualization template of the map.

15. The storage device of claim 14, further comprising enabling a street-level interactive map visualization of the location or an aerial-level interactive map visualization of the location.

16. The storage device of claim 14, further comprising identifying features of the location and selecting a visualization template of the plurality of visualization templates based on the features.

17. The storage device of claim 14, further comprising mapping a visualization template of the plurality of visualization templates to an axis of a display.

18. The storage device of claim 14, further comprising navigating between the plurality of visualization templates in response to an interactive tapping, movement beyond a template edge, or movement orthogonal to the template.