Geotagging photographs using annotations

• US 7,991,283 B2
• Filed: 09/30/2008
• Issued: 08/02/2011
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mishra, Pragyana K., Ofek, Eyal, Chen, Billy
• Primary Examiner(s): Laballe; Clayton E
• Assistant Examiner(s): Fenwick; Warren K
• Application Number:
  US12/242,739
• Publication Number:
  US 20100080551A1
• Time in Patent Office: 1,036 Days
• Field of Search: 396/306, 396/310, 396315-318
• US Class Current: 396/310
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G03B 17/24 with means for separately producing marks on the film, e.g. title, time of exposure
• CPC Class Codes: G03B 2217/246 Details of the markings
• CPC Class Codes: G03D 15/001 Counting; Classifying; Marking
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

Labels of elements in images may be compared to known elements to determine a region from which an image was created. Using this information, the approximate image position can be found, additional elements may be recognized, labels may be checked for accuracy and additional labels may be added.

Claims

1. A method of geotagging photographs using annotations comprising:

reviewing an image for a labeled element wherein the labeled element comprises one or more selected from a group comprising street signs, buildings, billboards, cars, transportation elements;

reviewing the image for an additional labeled element;

using the global position of the elements, the relative size of the elements and a width of the image to estimate a focal length of the image;

evaluating a known element data store to determine whether a known element in the known element data store is sufficiently similar to the labeled element;

if the data store contains a sufficiently similar known element to the labeled element, obtaining the global position of the known element;

comparing the additional labeled element to the known element data store to determine if a sufficiently similar additional known element to the additional labeled element is in the data store;

if a sufficiently similar element to the additional labeled element is in the known element data store, obtaining the global position of the additional known element;

evaluating the global position of the known element and the additional known element to determine an image region wherein the image region comprises a geographic area from which the image was created;

adding additional labels to the image by using geospatial data and building recognition to add labels to additional recognized buildings; and

if the labeled element and additional labeled element are accurately labeled, storing the image as an approved tagged image.

2. The method of claim 1, where the known element data store comprises element names and related global positions of the known elements.

3. The method of claim 1, further comprising if the labeled element or additional labeled element is not sufficiently similar to the known elements in the known element data store, ignoring the labeled element or additional labeled element.

4. The method of claim 1, further comprising repeating the method until all labeled elements are evaluated in view of the known element data store.

5. The method of claim 4, further comprising using additional labeled elements to further specify the image region.

6. The method of claim 5, further comprising using half space culling to further specify the image region.

7. The method of claim 5, further comprising using a scale of elements in the image to further specify the image region.

8. The method of claim 5, further comprising further specify the image region by evaluating regions from which it is possible to take an image.

9. The method of claim 1, further comprising adjusting the labeled elements to have labels that exactly match the labels in the known element data store.

10. The method of claim 1, further comprising determining if the labeled element and additional labeled element are accurately labeled by evaluating the relative position of the global position of the known element and the additional known element from a perspective of the image region.

11. The method of claim 1, wherein global position comprises one selected from a group comprising latitude and longitude and global positioning location information.

12. The method of claim 1, further comprising using one selected from a group comprising image metadata and image headers to further determine the focal length and image angle.

13. The method of claim 1, wherein the user selects the elements desired to be labeled.

14. A non-transitory computer storage medium comprising:

computer executable instructions to configuring a processor, the computer executable instructions comprising instructions for geo-tagging photographs using annotations, the instructions comprising instructions for;

Reviewing an image for a labeled element wherein the labeled element comprises one or more selected from a group comprising street signs, buildings, billboards, cars, transportation elements;

Reviewing the image for an additional labeled element;

Using the global position of the elements, the relative size of the elements and a width of the image to estimate a focal length of the image;

Evaluating a known element data store to determine whether a known element in the known element data store is sufficiently similar to the labeled element;

If the data store contains a sufficiently similar known element to the labeled element, obtaining the global position of the known element;

Comparing the additional labeled element to the known element data store to determine if a sufficiently similar additional known element to the additional labeled element is in the data store;

If a sufficiently similar element to the additional labeled element is in the known element data store, obtaining the global position of the additional known element;

Evaluating the global position of the known element and the additional known element to determine an image region wherein the image region comprises a geographic area from which the image was created;

Adding additional labels to the image by using geospatial data and building recognition to add labels to additional recognized buildings; and

If the labeled element and additional labeled element are accurately labeled, storing the image as an approved tagged image.

15. The non-transitory computer storage medium of claim 14, where the known element data store comprises element names and related global positions of the known elements.

16. The non-transitory computer storage medium of claim 14, further comprising computer executable instructions for repeating the method until all labeled elements are evaluated in view of the known element data store.

17. The non-transitory computer storage medium of claim 16, further comprising computer executable instructions for further specifying the image region comprising at least one selected from a group comprising:

using additional labeled elements;

using half space culling;

using a scale of elements in the image; and

evaluating regions from which it is possible to take an image.

18. The non-transitory computer storage medium of claim 14, further comprising computer executable instructions for determining if the labeled element and additional labeled element are accurately labeled by evaluating the relative position of the global position of the known element and the additional known element from a perspective of the image region.

19. The non-transitory computer storage medium of claim 14, wherein global position comprises one selected from a group comprising latitude and longitude and global positioning location information.

20. The non-transitory computer storage medium of claim 14, further comprising computer executable instructions for using one selected from a group comprising image metadata and image headers to further determine the focal length and image angle.