Applications of three-dimensional environments constructed from images

• US 7,712,052 B2
• Filed: 07/31/2006
• Issued: 05/04/2010
• Est. Priority Date: 07/31/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Szeliski, Richard S., Aguera y Arcas, Blaise H., Kasperkiewicz, Tomasz S. M.
• Primary Examiner(s): Hong; Stephen S
• Assistant Examiner(s): ROBINSON, MARSHON L
• Application Number:
  US11/461,315
• Publication Number:
  US 20080028341A1
• Time in Patent Office: 1,373 Days
• Field of Search: 715/205, 715/854, 345/419
• US Class Current: 715/854
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

Applications that extend the basic concept of determining spatial relationships between images and generating a model allowing users to experience the images in a 3D environment are provided. One application is directed to the creation of documents that may be readily shared, viewed, and edited, wherein the documents include references to underlying content comprising a 3D environment and images positioned therein and captures connections between images. Another application relates to ranking images based on user viewing information, such as the number of users who have viewed an image and how long an image was viewed. Yet another application is directed to analyzing the spatial density of images within a 3D environment to determine points of user interest. A further application relates to using spatially indexed images to navigate between webpages and 3D environments.

Claims

1. One or more computer-readable media having computer-useable instructions embodied thereon that, when executed by at least one processor, cause the at least one processor to perform a method for creating a document, the method comprising:

receiving a selection of a subset of two-dimensional (2D) images from a plurality of 2D images of a geographically contiguous area in the real-world, wherein the subset of 2D images is selected from a three-dimensional (3D) environment comprising a 3D model automatically generated from the plurality of 2D images, the 3D environment and 3D model representing the geographically contiguous area in the real-world corresponding with the plurality of 2D images, and wherein each of the plurality of 2D images is automatically located within the 3D environment based on spatial information derived from each of the plurality of 2D images;

identifying connections between images within the subset of 2D images, the connections defining one or more paths through the 3D environment by specifying navigation between images within the subset of 2D images within the 3D environment in accordance with the connections; and

creating a document containing a reference to each image within the subset of 2D images and an identification of the connections between images within the subset of 2D images.

2. The one or more computer-readable media of claim 1, wherein the plurality of 2D images comprise at least one of a photograph, a video, and an artwork image.

3. The one or more computer-readable media of claim 1, wherein the connections between images within the subset of 2D images are based on at least one of a spatial relationship, a subject relationship, a time relationship, and an author relationship.

4. The one or more computer-readable media of claim 1, wherein the instructions for creating the document include instructions for adding new 2D images to the 3D environment.

5. The one or more computer-readable media of claim 1, wherein the method further comprises associating additional content with the document, the additional content comprising at least one of an ambient audio, an audio narration, and an annotation.

6. The one or more computer-readable media of claim 5, wherein the document further defines a connection between at least one of an ambient audio, an audio narration, and an annotation with at least a portion of the one or more paths through the 3D environment.

7. The one or more computer-readable media of claim 1, wherein the one or more paths through the 3D environment comprise at least one of:

a linear path through the 3D environment; and

a plurality of paths through the 3D environment having one or more decision points between the plurality of paths.

8. The one or more computer-readable media of claim 1, wherein identifying connections between images within the subset of 2D images comprises automatically determining connections between images based on user viewing information associated with users viewing the images in the 3D environment.

9. The one or more computer-readable media of claim 1, wherein the computer-useable instructions further comprise at least one of instructions for editing the document, instructions for viewing the document, and instructions for sharing the document.

10. The one or more computer-readable media of claim 9, wherein the instructions for editing the document include instructions for editing the document without altering the 3D environment.

11. The one or more computer-readable media of claim 9, wherein the instructions for editing the document include instructions for adding new content to the 3D environment.

22. The one or more computer-readable media of claim 1, wherein identifying connections between images within the subset of 2D images comprises automatically determining connections between images based on the spatial density of the images in the 3D environment.

12. One or more computer-readable media having computer-useable instructions embodied thereon that, when executed by at least one processor, cause the at least one processor to perform a method of ranking images, the method comprising:

receiving user viewing information associated with each of a plurality of images of a geographically contiguous area in the real-world, each of the images being automatically positioned in a three-dimensional (3D) environment based on spatial information derived from each of the images, the 3D environment comprising a 3D model having been automatically generated from the plurality of images and representing the geographically contiguous area in the real-world;

receiving user navigation information associated with user navigations through the 3D environment;

automatically defining one or more oaths through the 3D environment based on the user navigation information; and

ranking at least a portion of the plurality of images based on the user viewing information associated with users viewing the images within the 3D environment to provide one or more image rankings.

13. The one or more computer-readable media of claim 12, wherein the user viewing information for an image includes information associated with at least one of a number of users who have viewed the image, a user dwell time for the image, and information associated with an author of the image.

14. The one or more computer-readable media of claim 12, wherein the method further comprises controlling the presentation of images based on the one or more image rankings.

15. The one or more computer-readable media of claim 12, wherein the method further comprises automatically defining one or more connections between images based on the one or more image rankings.

16. The one or more computer-readable medium of claim 12, wherein the user navigation information includes information regarding paths taken by users when navigating through the 3D environment.

17. One or more computer-readable media having computer-useable instructions embodied thereon that, when executed by at least one processor, cause the at least one processor to perform a method of using images to determine points of user interest, the method comprising:

providing a three-dimensional (3D) environment having a plurality of images of a geographically contiguous area in the real-world, wherein the 3D environment comprises a 3D model that is automatically generated based on the plurality of images and represents the geographically contiguous area in the real-world, each of the images being automatically positioned in the 3D environment based on spatial information derived from each of the images;

determining connections between images based on a spatial density of the images within the 3D environment;

defining paths through the 3D environment based on the connections between the images determined based on the spatial density of the images within the 3D environment; and

determining points of user interest in the geographically contiguous area in the real-world, by analyzing the spatial density of the images within the 3D environment.

18. The one or more computer-readable media of claim 17, further comprising controlling the presentation of images based on the spatial density of the images within the 3D environment.

19. One or more computer-readable media having computer-useable instructions embodied thereon that, when executed by at least one processor, cause the at least one processor to perform a method for navigating between a first image in a webpage and a three-dimensional (3D) environment having a plurality of images of a geographically contiguous area in the real-world, wherein the 3D environment comprises a 3D model that is automatically generated based on the plurality of images and represents the geographically contiguous area in the real-world, the method comprising:

presenting the first image within the webpage, the first image including a hyperlink to the 3D environment, the 3D environment having the plurality of images automatically positioned therein based on spatial information derived from each of the images, the 3D environment further having the first image positioned therein based on spatial information derived from the first image;

receiving a user selection of the first image within the webpage; and

responsive to the user selection of the first image within the webpage, presenting the 3D environment.

20. The one or more computer-readable media of claim 19, wherein the 3D environment is presented from a position within the 3D environment based on the spatial information associated with the first image.

21. The one or more computer-readable media of claim 19, wherein the method further comprises:

navigating to a second image in the 3D environment, wherein the second image includes a hyperlink to a second webpage, and wherein the second image is included within the second webpage;

receiving a user selection of the second image within the 3D environment; and

responsive to the user selection of the second image within the 3D environment, presenting the second webpage.