CONSTRUCTION OF PHOTO TRIP PATTERNS BASED ON GEOGRAPHICAL INFORMATION

• US 20110066588A1
• Filed: 09/16/2009
• Published: 03/17/2011
• Est. Priority Date: 09/16/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Arase, Yuki, Xie, Xing
• Granted Patent: US 8,626,699 B2
• Time in Patent Office: Days
• US Class Current: 706/58
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

Techniques for reconstructing photo trip patterns from geo-tagged photos are described. Photo trip patterns are reconstructed by mining geo-tagged photos from the Web or a data storage and segmenting the photos based on at least the geographical identification information associated with the photos. Mining semantics of each photo trip pattern may also be performed using tags associated with the photos.

Claims

1. A method comprising:

identifying a plurality of sets of metadata associated with a plurality of photos, each set of metadata being associated with a respective one of the photos and including at least geographical information related to a location where the respective photo was captured; and

determining photo trip patterns from the plurality of photos based on the plurality sets of metadata, each photo trip pattern being representative of a set of visited locations and associated transition times.

2. The method of claim 1, wherein each set of metadata further includes:

time information related to date and time on which the respective photo was captured;

a photo identification associated with the respective photo;

a user identification associated with a photo owner by whom the respective photo was captured; and

a tag associated with the respective photo having description of the location where the photo was captured.

3. The method of claim 1, wherein determining photo trip patterns comprises:

segmenting the plurality of photos into subsets of photos based on at least the geographical information related to each of the photos.

4. The method of claim 3, wherein segmenting the plurality of photos into subsets of photos comprises:

determining a respective location gap between the location where the respective photo was captured and a reference location for each of the plurality of photos; and

grouping photos having respective location gaps that fall within a respective threshold location gap range into a respective subset of photos.

6. The method of claim 1, wherein each set of metadata associated with a respective one of the photos further includes time information related to date and time on which the respective photo was captured, and wherein determining photo trip patterns comprises:

determining a respective location gap between the location where the respective photo was captured and a reference location for each of the plurality of photos;

determining a respective transition time gap between the last photo captured at a first location and the first photo captured at a second location visited after the first location for each pair of consecutively visited locations; and

grouping the plurality of photos into subsets of photos based on the location gaps and the transition time gaps, each subset photos related to a respective one of the plurality of locations and having a transition time gap with another subset greater than a threshold transition time gap.

7. The method of claim 6, wherein determining photo trip patterns further comprises:

identifying a sequence of locations as a photo trip pattern, each location having at least one subset of photos associated therewith and being separated from another location by a respective transition time gap greater than the threshold transition time gap.

8. The method of claim 1, wherein each set of metadata associated with a respective one of the photos further includes a tag associated with the respective photo having description of the location where the photo was captured, and wherein determining photo trip patterns further comprises:

identifying semantics associated with each photo trip pattern based on the respective tag associated with each of the photos.

9. The method of claim 8, wherein identifying semantics associated with each photo trip pattern based on tags associated with the photos comprises:

determining a term frequency for each tag as a number of times the respective tag is related to photos captured in the plurality of locations;

determining an inverse document frequency for each tag as a ratio of a number of total photos of the plurality of photos to a number of the photos having the respective tag associated therewith; and

determining a user frequency for each tag as a ratio of a number of photo owners who use the respective tag to describe at least one of the plurality of photos to a number of total photo owners of the plurality of photos.

10. The method of claim 9 further comprising:

determining a score for each tag using the respective term frequency, inverse document frequency, and user frequency.

11. The method of claim 8, wherein each location where each of the photos was captured is represented by a hierarchy of geographical regions including at least a first level of geographical region and a second level of geographical region that includes the first level of geographical region, and wherein identifying semantics associated with each photo trip pattern based on the respective tag associated with each of the plurality of photos comprises identifying semantics associated with each photo trip pattern by considering tags at the first level of geographical region.

12. The method of claim 1, wherein the geographical information includes latitudinal and longitudinal information of the location where the respective photo was captured.

13. The method of claim 12 further comprising:

converting the latitudinal and longitudinal information related to each photo a hierarchical representation of the respective location, the hierarchical representation including at least a geographical region of a first level and a geographical region of a second level that encompasses the geographical region of the first level.

14. The method of claim 13, wherein the hierarchical representation of the location includes a name of a city where the respective photo was captured, a name of a state in which the city is located, and a name of a country in which the state is located.

5. The method of 1, wherein each set of metadata associated with a respective one of the photos further includes time information related to date and time on which the respective photo was captured, and wherein determining photo trip patterns comprises:

segmenting the plurality of photos into subsets of photos based on the geographical information and the time information related to each of the photos.

15. A method comprising:

segmenting a plurality of photos into subsets of photos based on geographical information and time information related to each of the photos;

identifying photo trip patterns, each of which having a sequence of locations, each location having at least one of the subsets of photos associated therewith and being separated from another location by a respective transition time gap greater than a threshold transition time gap; and

identifying semantics associated with each photo trip pattern based on tags associated with the plurality of photos, each of the tags being associated with at least a respective one of the plurality of photos and describing the location where the respective photo was captured.

16. The method of claim 15, wherein segmenting a plurality of photos into subsets of photos based on geographical information and time information related to each of the photos comprises:

determining a respective location gap between the location where the respective photo was captured and a reference location for each of the plurality of photos;

determining a respective transition time gap between a last photo captured at a first location and a first photo captured at a second location visited after the first location for each pair of consecutively visited locations; and

grouping the plurality of photos into subsets of photos based on the location gaps and the transition time gaps, each subset of photos related to a respective one of the plurality of locations.

17. The method of claim 15, wherein identifying semantics associated with each photo trip pattern based on tags associated with the plurality of photos comprises:

determining a term frequency for each tag as a number of times the respective tag is related to photos captured in the plurality of locations;

determining an inverse document frequency for each tag as a ratio of a number of total photos of the plurality of photos to a number of the photos having the respective tag associated therewith; and

determining a user frequency for each tag as a ratio of a number of photo owners who use the respective tag to describe at least one of the plurality of photos to a number of total photo owners of the plurality of photos.

18. The method of claim 15 further comprising:

converting latitudinal and longitudinal information included in the geographical information related to each of the photos into a hierarchical representation of the location where the respective photo was captured, the hierarchical representation including at least a geographical region of a first level and a geographical region of a second level that encompasses the geographical region of the first level.

19. One or more computer readable media storing computer-executable instructions that, when executed, perform acts comprising:

identifying a plurality of sets of metadata associated with a plurality of photos, each set of metadata being associated with a respective one of the photos and including at least geographical information related to a location where the respective photo was captured; and

determining photo trip patterns from the plurality of photos based on the plurality sets of metadata, each photo trip pattern being representative of a set of visited locations and associated typical transition times.

20. The computer-readable media of claim 19, wherein the act of determining photo trip patterns from the plurality of photos based on the plurality sets of metadata comprises:

segmenting the plurality of photos into subsets of photos based on the geographical information related to each of the photos;

identifying the photo trip patterns wherein each location has at least one of the subsets of photos associated therewith; and

identifying semantics associated with each photo trip pattern based on tags associated with the plurality of photos, each of the tags being associated with a respective one of the plurality of photos and describing the location where the respective photo was captured.