System for logging life experiences using geographic cues

• US 8,972,177 B2
• Filed: 02/26/2008
• Issued: 03/03/2015
• Est. Priority Date: 02/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Zheng, Yu, Wang, Longhao, Xie, Xing, Zhang, Ruochi
• Primary Examiner(s): Holwerda, Stephen
• Application Number:
  US12/037,347
• Publication Number:
  US 20090216435A1
• Time in Patent Office: 2,562 Days
• Field of Search: 701/202, 701/209, 701/424, 701/429, 701/436, 701/454, 707/999.003, 707/999.004, 707/999.1, 707/741, 707/920, 707/E17.018
• US Class Current: 701/454
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G08G 1/096838 where the user preferences are taken into account or the user selects one route out of a plurality
• CPC Class Codes: G09B 29/007 using computer methods
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/4345 Extraction or processing of SI, e.g. extracting service information from an MPEG stream
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: Y10S 707/92 Navigation

Abstract

A system logs life experiences using geographic cues. The system variously provides a comprehensive life-logging tool for recording a plurality of different types of life events. In one implementation, the system receives a user'"'"'s GPS log files and multimedia content at a website. The system segments the GPS log files into geographic routes corresponding to user trips, and tags the multimedia content with geographic cues from the GPS log files. Then, the system indexes the geographic routes so that users can retrieve the geographic routes by browsing or by search techniques. The system displays animations of selected routes on a map, and displays the multimedia content at corresponding locations along the map route, as the route is replayed. The system also provides browsing and spatial and temporal techniques to search a person'"'"'s travels and can provide graphical displays of the person'"'"'s activity statistics.

Claims

1. A method, comprising:

receiving, at a website, a location information file from a user;

receiving multimedia content from the user;

segmenting the location information file into user trips;

inferring transportation mode of the user trips based at least in part on elapsed time between points in the location information file;

associating the multimedia content with the user trips by tagging locations in the user trips to the multimedia content, wherein tagging locations in the user trips to the multimedia content enables a search for particular multimedia content to return a corresponding user trip; and

displaying the user trips on a map, with indications of the inferred transportation mode and the associated multimedia content.

2. The method as recited in claim 1, further comprising:

displaying an animation of one of the user trips on the displayed map;

displaying an icon representing the user at a location on a user trip, the icon having an appearance representing a transportation mode used by the user at the location; and

moving the icon along the user trip.

3. The method as recited in claim 2, further comprising displaying the multimedia content when the icon moves to a location on the user trip with which multimedia content is associated.

4. The method as recited in claim 1, further comprising receiving a search criterion from the user, wherein the search criterion retrieves user trips that meet the search criterion from a spatial-temporal index;

wherein the user trips are listed as search results on the website.

5. The method as recited in claim 4, wherein the search criterion comprises one or both of:

a spatial search criterion created via the user selecting a geographic area on the map in order to retrieve search results comprising user trips within the selected geographic area;

or

a temporal search criterion created via the user selecting a time interval in order to retrieve search results comprising user trips made within the selected time interval.

6. The method as recited in claim 1, further comprising recommending geographic routes and local features, wherein when a user in a given location requests a recommendation for a geographic route or a local feature, popular geographic routes and popular local features are retrieved from the public knowledge database and displayed to the user;

and

wherein a public knowledge database is mined for personal preferences of individual users.

7. The method as recited in claim 6, further comprising interfacing with mobile devices, wherein when a mobile device in a given location requests a recommendation for a geographic route or a local feature, popular geographic routes and popular local features are retrieved from the public knowledge database and displayed to the mobile device.

8. The method as recited in claim 1, further comprising displaying ancillary information calculated in relation to the user trip or from a public knowledge database to correspond with times and places along the user trips;

wherein the ancillary information includes one of a weather report, a temperature, a distance of the user trip, a duration of the user trip, a start and an ending time of the user trip, a number of multimedia images and videos associated with the user trip, a number of segments of the user trip, modes of transportation used during the user trip, a latitude and longitude of a location on the user trip, an altitude of a location on the user trip, news associated with a time or a location of the user trip, various map views associated with locations along the user trip, or advertisements associated with locations along the user trip.

9. The method as recited in claim 2, further comprising:

deriving statistics from a user'"'"'s activity of the user indicated by the location information file;

creating a visual depiction of the statistics; and

displaying the visual depiction of the statistics to the user on the website or on a mobile device of the user.

10. The method as recited in claim 9, further comprising displaying statistics of the user'"'"'s activities, including one of:

an activity level for each day of the week;

a ratio of transportation modes used over a time interval;

a pattern of arrivals to or departures from a routine user location.

11. A system, comprising:

a web user interface for uploading location information files of a user and multimedia contents to a service;

a data processor to segment the location information files into geographic routes, to associate the multimedia contents to the geographic routes by tagging locations in the geographic routes to the multimedia contents, and to infer at least one transportation mode within a geographic route;

an indexing engine to index the geographic routes, the indexing engine dividing the geographic routes into trips, the trips separated by a threshold duration of time, the indexing engine dividing the trips into segments, the segments distinguished based on a transportation mode of the user; and

a visualization engine to display an animation of one of the indexed geographic routes on a map displayed on the web user interface, to display multimedia content associated with times and locations along the geographic routesand to display an icon representing the user at a location on the geographic routes, the icon having an appearance representing a transportation mode used by the user at the location.

12. The system as recited in claim 11, further comprising a route searching engine to receive a search criterion from the user, wherein the route searching engine uses the search criterion to retrieve geographic routes that meet the search criterion from a spatial-temporal index;

wherein the geographic routes are listed as search results on the web user interface.

13. The system as recited in claim 11, further comprising:

moving the icon along the geographic routes in forward and reverse directions along the geographic routes to allow a user to navigate the icon to select desired multimedia content; and

displaying the multimedia content when the icon moves to a location on the geographic routes with which multimedia content is associated.

14. The system as recited in claim 11, further comprising a recommendation engine for requesting geographic routes and local features, wherein when a user in a given location requests a recommendation for a geographic route or a local feature, then popular geographic routes and popular local features are retrieved from a public knowledge database;

and

wherein the public knowledge database is mined from personal preferences of individual users derived from the location information files uploaded by the individual users.

15. The system as recited in claim 14, further comprising a mobile devices interface, wherein when a mobile device in a given location requests a recommendation for a geographic route or a local feature, popular geographic routes and popular local features are retrieved from the public knowledge database and displayed to the mobile device.

16. The system as recited in claim 11, wherein the visualization engine displays ancillary information corresponding to times and places along a geographic route;

and

wherein the ancillary information includes one of a weather report, a temperature, a distance of a user trip, a duration of the user trip, a start and an ending time of the user trip, a number of multimedia images and videos associated with the user trip, a number of segments of the user trip, modes of transportation used during the user trip, a latitude and longitude of a location on the user trip, an altitude of a location on the user trip, news associated with a time or a location of the user trip, various map views associated with locations along the geographic route, or advertisements associated with locations along the geographic route.

17. The system as recited in claim 11, wherein the visualization engine derives statistics from a user'"'"'s activity indicated by the user'"'"'s uploaded location information files, creates a visual depiction of the statistics, and displays the visual depiction of the statistics to the user on the website or an a mobile device of the user.

18. The system as recited in claim 17, wherein the visualization engine displays statistics of the user'"'"'s activity, including one of:

an activity level for each day of the week;

a ratio of transportation modes used over a time interval;

or

a pattern of arrivals to or departures from a routine user location.

19. A system, comprising:

means for segmenting a location information file received from a user into geographic routes, the segmenting including dividing the location information file into several trips if a time interval between two consecutive location points exceeds a time threshold;

means for tagging multimedia contents received from the user with geographic locations from the location information file to associate the multimedia contents with the geographic routes; and

means for displaying an animation of a user trip on a map, wherein the animation displays cues to the multimedia content at geographic locations on the map, the means for displaying including;

means for displaying the geographic routes on the displayed map;

means for displaying an icon having a location representing a location of the user at a location on the geographic routes and having an appearance representing a transportation mode used by the user at the location;

means for moving the icon along the geographic routes in forward and reverse directions along the geographic routes to allow a user to navigate the icon to select desired multimedia content; and

means for displaying the multimedia content, when the icon moves to a location on the geographic routes with which multimedia content is associated.