Geotagging Photographs Using Annotations

• US 20100080551A1
• Filed: 09/30/2008
• Published: 04/01/2010
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ofek, Eyal, Chen, Billy, Mishra, Pragyana K.
• Granted Patent: US 7,991,283 B2
• Time in Patent Office: Days
• US Class Current: 396/310
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G03B 17/24 with means for separately producing marks on the film, e.g. title, time of exposure
• CPC Class Codes: G03B 2217/246 Details of the markings
• CPC Class Codes: G03D 15/001 Counting; Classifying; Marking
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

Labels of elements in images may be compared to known elements to determine a region from which an image was created. Using this information, the approximate image position can be found, additional elements may be recognized, labels may be checked for accuracy and additional labels may be added.

Claims

1. A method of geotagging photographs using annotations comprising:

Reviewing an image for a labeled element;

Reviewing the image for an additional labeled element;

Evaluating a known element data store to determine whether a known element in the known element data store is sufficiently similar to the labeled element;

If the data store contains a sufficiently similar known element to the labeled element, obtaining the global position of the known element;

Comparing the additional labeled element to the known element data store to determine if a sufficiently similar additional known element to the additional labeled element is in the data store;

If a sufficiently similar element to the additional labeled element is in the known element data store, obtaining the global position of the additional known element;

Evaluating the global position of the known element and the additional known element to determine an image region wherein the image region comprises a geographic area from which the image was created;

Adding additional labels to the image by using geospatial data and building recognition to add labels to additional recognized buildings; and

If the if the labeled element and additional labeled element are accurately labeled, storing the image as an approved tagged image.

2. The method of claim 1, where the known element data store comprises element names and related global positions of the known elements.

3. The method of claim 1, further comprising if the labeled element or additional labeled element is not sufficiently similar to the known elements in the known element data store, ignoring the labeled element or additional labeled element.

4. The method of claim 1, further comprising repeating the method until all labeled elements are evaluated in view of the known element data store.

5. The method of claim 4, further comprising using additional labeled elements to further specify the image region.

6. The method of claim 5, further comprising using half space culling to further specify the image region.

7. The method of claim 5, further comprising using a scale of elements in the image to further specify the image region.

8. The method of claim 5, further comprising further specify the image region by evaluating regions from which it is possible to take an image.

9. The method of claim 1, further comprising adjusting the labeled elements to have labels that exactly match the labels in the known element data store.

10. The method of claim 1, further comprising determining if the labeled element and additional labeled element are accurately labeled by evaluating the relative position of the global position of the known element and the additional known element from a perspective of the image region.

11. The method of claim 1, wherein global position comprises one selected from a group comprising latitude and longitude and global positioning location information.

12. The method of claim 1, further comprising using one selected from a group comprising image metadata and image headers to further determine the focal length and image angle.

13. The method of claim 1, further comprising using the global position of the elements, the relative size of the elements and a width of the image to estimate a focal length of the image.

14. The method of claim 1, wherein elements comprise one or more selected from a group comprising street signs, buildings, billboards, cars, transportation elements.

15. The method of claim 1, wherein the user selects the elements desired to be labeled.

16. The method of claim 1, wherein the image position is calculated using an energy function comprising:

E(c)=summation I(Pc(Li)−

Ni)̂

2

Wherein

c is a camera parameters desired (position, orientation, focal length);

P is a projection function;

L is landmark latitude; and

N is landmark longitude.

17. The method of claim 16, further comprising adding an additional constraint to the energy function to ensure that the camera position is within the region defined using half space culling.

19. The computer storage medium of claim 17, wherein the image position is calculated using an energy function comprising:

E(c)=summation I(Pc(Li)−

Ni)̂

2

Wherein

c is a camera parameters desired (position, orientation, focal length);

P is a projection function;

L is landmark latitude; and

N is landmark longitude.

20. The computer storage medium of claim 19, further comprising computer executable instructions for adding an additional constraint to the energy function to ensure that the camera position is within the region defined using half space culling.

18. A computer storage medium comprising computer executable instructions to configuring a processor, the computer executable instructions comprising instructions for geo-tagging photographs using annotations, the instructions comprising instructions for:

Reviewing an image for a labeled element;

Reviewing the image for an additional labeled element;

Evaluating a known element data store to determine whether a known element in the known element data store is sufficiently similar to the labeled element;

If the data store contains a sufficiently similar known element to the labeled element, obtaining the global position of the known element;

Comparing the additional labeled element to the known element data store to determine if a sufficiently similar additional known element to the additional labeled element is in the data store;

If a sufficiently similar element to the additional labeled element is in the known element data store, obtaining the global position of the additional known element;

Evaluating the global position of the known element and the additional known element to determine an image region wherein the image region comprises a geographic area from which the image was created;

Adding additional labels to the image by using geospatial data and building recognition to add labels to additional recognized buildings; and

If the if the labeled element and additional labeled element are accurately labeled, storing the image as an approved tagged image.