Geo-relevance for images

• US 8,326,048 B2
• Filed: 10/04/2007
• Issued: 12/04/2012
• Est. Priority Date: 10/04/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ofek, Eyal, Epshtein, Boris
• Primary Examiner(s): LEE, JOHN W
• Application Number:
  US11/867,053
• Publication Number:
  US 20090092277A1
• Time in Patent Office: 1,888 Days
• Field of Search: None
• US Class Current: 382/224
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

Images may be sorted and categorized by defining a frustum for each image and overlaying the frustums in two, three, or four dimensions to create a density map and identify points of interest. Images that contain a point of interest may be grouped, sorted, and categorized to determine representative images of the point. By including many images from different sources, common points of interest may be defined. Points of interest may be defined in two or three Euclidian dimensions, or may include a dimension of time.

Claims

1. A method performed at least in part by a computer processor, said method comprising:

determining a plurality of image frustums having an approximate point of origin and an approximate direction, each of said image frustums being determined from data derived from each of an individual image and representing a geographic area covered by said individual image;

arranging said plurality of image frustums based on said approximate point of origin and said approximate direction relative to one another in a geographic representation to form a relevance map showing one or more densities of coverage by said plurality of image frustums;

determining a first area of relevance from said relevance map based on said one or more densities of coverage of said relevance map; and

identifying a first group of images that include said first area of relevance.

2. The method of claim 1, said frustums further having at least one of a group composed of:

a view angle;

a depth;

a timestamp; and

a relevance function.

3. The method of claim 2, said depth being determined by at least one of a group composed of:

computing an approximate geographical distance from said point of origin to an object;

mapping said frustum to a known geographical model; and

mapping a first portion of a first image to a second portion of a second image.

4. The method of claim 1, said relevance map having a dimension of time.

5. The method of claim 1, said relevance map having three dimensions.

6. The method of claim 1, further comprising:

selecting a representative image from said first group.

7. The method of claim 6, said selecting a representative image being based on at least one of a group composed of:

overlap of said first area of relevance with said plurality of image frustums;

sharpness; and

resolution.

8. The method of claim 1 further comprising:

selecting a second area of relevance from said first group of said plurality of images; and

identifying a second group of said plurality of images that include said second area of relevance.

9. A computer readable medium not comprising a signal but comprising computer executable instructions adapted to perform the method of claim 1.

10. A system comprising:

a processor;

a frustum determination engine operating on said processor and adapted to determine a frustum of an image, using data derived from said image, said frustum comprising an approximate point of origin and an approximate direction and representing a geographic area covered by said image;

a mapping engine adapted to map a plurality of frustums relative to one another in a geographic representation onto a relevance map showing one or more densities of coverage by said plurality of mapped frustums; and

a grouping engine adapted to determine a first area of relevance from said relevance map and create a first group of images that include said first area of relevance based on said one or more densities of coverage of the relevance map.

11. The system of claim 10, said frustum determination engine adapted to determine at least one frustum by geopositional metadata for an image.

12. The system of claim 11, said geopositional metadata being determined by at least one of a group composed of:

user input; and

a geopositional detection system.

13. The system of claim 10, said frustum further having at least one of a group of parameters composed of:

a focal point;

a view angle;

a depth;

a timestamp; and

a relevance function.

14. The system of claim 13, at least one of said parameters in said group of parameters being determined by an image capture device.

15. The system of claim 10, said grouping engine further adapted to:

rank a plurality of images in said first group of images.

16. The system of claim 10, said grouping engine further adapted to:

select a representative image from said first group of images.

17. A method performed at least in part by a computer processor, said method comprising:

receiving a plurality of images, said images being created by a plurality of sources;

approximating a frustum for each of said plurality of images based on data derived from each respective one of said plurality of images, said frustum representing a geographic area covered by said individual images;

determining an area of relevance based on said frustums by

overlapping said plurality of approximated frustums relative to one another in a geographic representation to form a relevance map;

determining a density of coverage over said area of relevance on the relevance map based on a number of said plurality of frustums overlapping over said area of relevance; and

selecting a first group of said plurality of images based on said area of relevance.

18. The method of claim 17, said images being individual frames of a video sequence.

19. The method of claim 17 further comprising:

applying a relevance function to each of said frustums.

20. A computer readable medium not comprising a signal but comprising computer executable instructions adapted to perform the method of claim 17.