Navigating images using image based geometric alignment and object based controls

• US 20070110338A1
• Filed: 07/25/2006
• Published: 05/17/2007
• Est. Priority Date: 11/17/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation), University of Washington
• Original Assignee: Microsoft Corporation, University of Washington
• Inventors: Szeliski, Richard, Seitz, Steven, Snavely, Keith
• Granted Patent: US 8,160,400 B2
• Time in Patent Office: Days
• US Class Current: 382/305
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: G06V 20/39 Urban scenes

Abstract

Over the past few years there has been a dramatic proliferation of digital cameras, and it has become increasingly easy to share large numbers of photographs with many other people. These trends have contributed to the availability of large databases of photographs. Effectively organizing, browsing, and visualizing such .seas. of images, as well as finding a particular image, can be difficult tasks. In this paper, we demonstrate that knowledge of where images were taken and where they were pointed makes it possible to visualize large sets of photographs in powerful, intuitive new ways. We present and evaluate a set of novel tools that use location and orientation information, derived semi-automatically using structure from motion, to enhance the experience of exploring such large collections of images.

Claims

1. A system for navigating among a plurality of images, comprising:

an electronically generated interactive display;

a representation of a three-dimensional (3D) geometry on said display, said 3D geometry comprising at least one translucent projection of a digital photograph onto said 3D geometry, and wherein said 3D geometry is configured to be navigated by a user;

at least one selectable camera position associated with a digital photograph of a location depicted by said 3D geometry.

2. The system of claim 1, wherein upon selection of a selectable camera position, said system is configured to display an enlarged view of the associated digital photograph.

3. The system of claim 2, wherein upon selection of a selectable camera position, said system is further configured to display a plurality of selectable digital photographs of said location depicted by said 3D geometry.

4. The system of claim 3, wherein upon selection of a selectable digital photograph, said system is configured to display a sequence of transitional views of said location depicted by said 3D geometry prior to displaying said digital photograph.

5. The system of claim 3, wherein said sequence of transitional views of said 3D geometry are generated by projecting two images onto a common plane and cross-fading between the projected images.

6. The system of claim 1, wherein said at least one translucent projection of a digital photograph comprises a translucent projection of a combined plurality of digital photographs that are combined using alpha blending.

7. The system of claim 1, said 3D geometry further comprising a plurality of points and lines.

8. The system of claim 1, further comprising a tool for modifying a thickness of said plurality of lines.

9. The system of claim 1, further comprising a map view of said 3D geometry and camera viewpoints.

10. The system of claim 9, further comprising a selectable camera position in said map view, wherein upon selection of said selectable camera position, the system is configured to display a digital photograph associated with said selectable camera location in a main location of a user interface.

11. A system for navigating among a plurality of images, comprising:

an electronically generated interactive display;

a first digital photograph displayed at a main location within said display, wherein said first digital photograph comprises first image content;

at least one selectable alternate digital photograph displayed at a second location within said display, wherein said alternate digital photograph comprises image content that is overlapping or proximal to said first image content;

wherein upon selection of said selectable alternate digital photograph, said system is configured to display transitional image content in said main location prior to displaying said alternate digital photograph in said main location.

12. The system of claim 11, wherein said second location within said display is a location that is overlaid on top of the first digital photograph such that image content in said selectable alternate digital photograph overlaps corresponding first image content.

13. The system of claim 11, further comprising a search tool for finding alternate digital photographs comprising image content with a close-up view of at least one portion of said first image content.

14. The system of claim 11, further comprising a search tool for finding alternate digital photographs comprising image content that is overlapping substantially all of said first image content without substantial additional image content.

15. The system of claim 14, wherein said system is configured to sequence said alternate digital photographs according to a time when said alternate digital photographs were captured.

16. The system of claim 11, further comprising a search tool for finding alternate digital photographs comprising image content that is overlapping substantially all of said first image content and further comprising substantial additional image content, thereby providing a zoomed-out view of said first image content.

17. The system of claim 11, wherein said at least one selectable alternate digital photograph comprises at least two selectable alternate digital photographs, a first alternate digital photograph comprising image content proximal to a left side of said first image content, and a second alternate digital photograph comprising image content proximal to a right side of said first image content.

18. The system of claim 11, further comprising a selectable slideshow tool for causing said system to sequentially display a plurality of digital photographs in said main location, said plurality of digital photographs varying in either a time when said digital photographs were captured or a viewpoint from which said digital photographs were captured.

19. The system of claim 18, wherein said plurality of digital photographs are geometrically stabilized by projecting them onto a common plane in the main location within said display.

20. A system for navigating among a plurality of images, comprising:

an electronically generated interactive display;

an image located within said display, wherein said image comprises image content;

a plurality of stored alternate images also comprising said image content;

a tool for searching among said stored alternate images for a best view of a selected portion of said image content, wherein a best view is defined as at least one of;

a view from a direction that is most closely perpendicular to a plane associated with said image content, a view in which a greatest number of points associated with the selected portion of said image content are visible, and a view comprising a greatest amount of detail associated with said image content.

21. The system of claim 20, wherein said system is configured to score said stored alternate images according to best view of said selected portion of said image content.

22. The system of claim 21, wherein said system is configured to display a plurality of said stored alternate images in a sequence corresponding to a score value associated with each displayed image.