Analysis of images located within three-dimensional environments

• US 9,122,368 B2
• Filed: 03/11/2010
• Issued: 09/01/2015
• Est. Priority Date: 07/31/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Szeliski, Richard S., Aguera y Arcas, Blaise H., Kasperkiewicz, Tomasz S. M.
• Primary Examiner(s): Hong, Stephen
• Assistant Examiner(s): ROBINSON, MARSHON L
• Application Number:
  US12/722,280
• Publication Number:
  US 20100169838A1
• Time in Patent Office: 2,000 Days
• Field of Search: 345/419, 715/205, 715/854
• US Class Current: 1/1
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

Images are analyzed within a 3D environment that is generated based on spatial relationships of the images and that allows users to experience the images in the 3D environment. Image analysis may include ranking images based on user viewing information, such as the number of users who have viewed an image and how long an image was viewed. Image analysis may further include analyzing the spatial density of images within a 3D environment to determine points of user interest.

Claims

1. A computer-implemented method being performed by one or more computing devices including at least one processor, for ranking images, the method comprising:

receiving user viewing information associated with each of a plurality of images of a geographically contiguous area, the plurality of images being a collection of photographs obtained from a variety of sources including at least one person, the geographically contiguous area representing a scene comprising adjacent objects or different perspectives of a common object in the real world, each of the images being automatically positioned in a three-dimensional (3D) environment based on spatial information corresponding to the geographically contiguous area and derived from each of the images, the 3D environment comprising a 3D model having been automatically generated from the plurality of images and representing a model of the real world scene; and

ranking at least a portion of the plurality of images based on the user viewing information associated with users viewing the images within the 3D environment to provide one or more image rankings.

2. The method of claim 1, wherein the user viewing information for an image includes information associated with at least one of the following:

a number of users who have viewed the image; and

a user dwell time for the image indicating how long a user has viewed the image.

3. The method of claim 1, further comprising controlling presentation of at least a subset of the plurality of images based on the one or more image rankings.

4. The method of claim 3, wherein controlling presentation of at least a subset of the plurality of images based on the one or more image rankings includes selecting a subset of higher ranking images for presentation and removing from presentation a subset of lower ranking images.

5. The method of claim 4, further comprising periodically selecting one or more images from the subset of lower ranking images for presentation.

6. The method of claim 1, further comprising automatically defining one or more connections between images based on the one or more image rankings.

7. The method of claim 1, further comprising receiving user navigation information associated with user navigations through the 3D environment.

8. The method of claim 7, wherein the user navigation information includes information regarding paths taken by users when navigating through the 3D environment, the paths representative of images users have viewed in the 3D environment and an order in which the images were viewed.

9. A computer-implemented method being performed by one or more computing devices including at least one processor, for using images to determine points of user interest, the method comprising:

providing a three-dimensional (3D) environment having a plurality of images of a geographically contiguous area in the real world, the plurality of images being a collection of photographs obtained from a variety of sources including at least one person, the geographically contiguous area representing a scene comprising adjacent objects or different perspectives of a common object in the real world, wherein the 3D environment comprises a 3D model that is automatically generated based on the plurality of images and represents a model of the real world scene, each of the images being automatically positioned in the 3D environment based on spatial information corresponding to the geographically contiguous area and derived from each of the images; and

determining points of user interest in the geographically contiguous area in the real world by analyzing the spatial density of the images within the 3D environment.

10. The method of claim 9, further comprising controlling the presentation of images based on the spatial density of the images within the 3D environment.

11. The method of claim 10, wherein controlling presentation of images based on the spatial density of the images within the 3D environment comprises selecting a first subset of images for presentation and removing from presentation a second subset of images.

12. The method of claim 9, further comprising determining connections between images based on the spatial density of the images within the 3D environment.

13. The method of claim 9, further comprising determining paths traveled by individuals in the geographically contiguous area in the real world by analyzing the spatial density of the images within the 3D environment.

14. One or more computing devices having at least one processor and one or more computer-readable media, the one or more computing devices comprising:

a three-dimensional (3D) environment generating component that generates a 3D environment from a plurality of images of a geographically contiguous area in the real world, the plurality of images being a collection of photographs obtained from a variety of sources including at least one person, the geographically contiguous area representing a scene comprising adjacent objects or different perspectives of a common object in the real world, the 3D environment comprising a 3D model representing a model of the real world scene generated from the plurality of images, wherein the 3D environment generating component positions each image of at least a portion of the plurality of images within the 3D environment based on spatial information corresponding to the geographically contiguous area and derived from each image to organize the photographs spatially and create transitions between them that preserve their geometric relationships; and

an analysis component that analyzes information regarding images located within the 3D environment.

15. The one or more computing devices of claim 14, wherein the analysis component includes means for analyzing user viewing information associated with users viewing images within the 3D environment.

16. The one or more computing devices of claim 15, wherein the user viewing information for an image includes information associated with at least one of the following:

a number of users who have viewed the image; and

a user dwell time for the image indicating how long a user has viewed the image.

17. The one or more computing devices of claim 15, wherein the analysis component further comprises means for controlling presentation of at least a subset of the plurality of images based on the user viewing information.

18. The one or more computing devices of claim 14, wherein the analysis component includes means for analyzing spatial density information corresponding with the spatial density of images within the 3D environment.

19. The one or more computing devices of claim 18, wherein analyzing the spatial density information includes determining points of user interest in the geographically contiguous area in the real world by analyzing the spatial density of images within the 3D environment.

20. The one or more computing devices of claim 18, wherein analyzing the spatial density information includes determining paths traveled by individuals in the geographically contiguous area in the real world by analyzing the spatial density of images within the 3D environment.