Semantic place labels

• US 9,443,326 B2
• Filed: 12/10/2013
• Issued: 09/13/2016
• Est. Priority Date: 12/10/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Krumm, John C., Rouhana, Dany, Chang, Ming-Wei, Kansal, Aman, Choudhury, Piali
• Primary Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US14/101,937
• Publication Number:
  US 20150161439A1
• Time in Patent Office: 1,008 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

The subject disclosure is directed towards automatically labeling location-related information such as corresponding to GPS data or the like with a semantic label. A classifier trained with machine learning is provided with feature data corresponding to the location-related information and other features, such as user demographics data of a person associated with location-related information. The semantic label is received from the classifier, and associated with the location-related information. Other features may be used, such as other egocentric features corresponding to a person'"'"'s particular visit, features from a sequence of visits, and/or features from other user information. The semantic label may be used to trigger an action, label a location on a map or the like, and so on.

Claims

1. In a computing environment, a method performed at least in part on at least one processor, the method comprising:

receiving feature data corresponding to location-related information of an individual visit, the location-related information associated with a first computing device, the feature data including user demographics data of a person associated with the first computing device and at least one of arrival day of week, arrival time of day, visit midpoint time of day, departure time of day, duration of visit, season of year, holiday data, and nearby business features;

computing an inference of a semantic label for a place represented by the location-related information for the individual visit using a semantic label classifier and the received feature data, the inference of the semantic label based at least in part on the user demographics data, such that the inferred semantic label for the place is customized to the person associated with the user demographics data and the individual visit; and

associating the inferred semantic label with the place represented by the location-related information for the individual visit; and

automatically performing an action at the first computing device or at another computing device based on the inferred semantic label.

2. The method of claim 1 wherein providing the feature data comprises providing sequence-related feature data to the classifier.

3. The method of claim 1 wherein associating the inferred semantic label with the place represented by the location-related information comprises inferring the semantic label for an individual visit of at least two visits of a sequence of visits.

4. The method of claim 1 wherein performing the action based on the inferred semantic label further comprises:

taking automated action based upon the inferred semantic label, including at least one of;

providing the inferred semantic label in association with a notification or presenting the inferred semantic label on a user interface.

5. The method of claim 1 further comprising:

maintaining the inferred semantic label in association with historical location data.

6. The method of claim 1 wherein computing the inference of the semantic label for the place represented by the location-related information further comprises:

computing a set of probabilities for individual visits by the person to the place represented by the location-related information, such that an individual inferred semantic label is associated with individual probability data corresponding to the person associated with the user demographic data and an individual visit.

7. The method of claim 1, wherein performing the action comprises changing one or more of a ring tone state of the first computing device, a calendar state of the first computing device, a connectivity state of the first computing device, a security state of a second computing device remote from the first computing device, a power state of the second computing device, or an authentication state of the second computing device.

8. The method of claim 1, further comprising:

receiving a second inferred semantic label associated with a location of a second computing device; and

presenting a notification of the location of the second computing device on a user interface of the first computing device.

9. The method of claim 1, further comprising:

receiving second location-related information associated with a second computing device;

based on the received second location-related information, obtaining second feature data corresponding to the second location-related information, the second feature data including user demographics data of a second person associated with the second computing device;

associating a second semantic label with the second location-related information; and

presenting a notification of the second semantic label on a user interface of the first computing device.

10. The method of claim 1, further comprising:

receiving user feedback based on the automatically performed action; and

training the semantic label classifier at least in part using the received user feedback.

11. A system comprising:

at least one processor; and

a memory device having computer-executable instructions, the memory device coupled to the at least one processor such that the at least one processor is configured to execute the computer-executable instructions to;

receive feature data and location-related information corresponding to a visit, the feature data including user demographics data and at least one of arrival day of week, arrival time of day, visit midpoint time of day, departure time of day, duration of visit, season of year, holiday data, and nearby business features;

compute an inference of a semantic label for a place represented by the location-related information for that visit using a semantic label classifier and the received feature data, the inference of the semantic label based at least in part on the user demographics data, such that the inferred semantic label is customized to a person associated with the user demographics data and that visit;

label the visit with the inferred semantic label based upon features of the visit; and

upon labeling the visit with the inferred semantic label, automatically perform an action that corresponds to the inferred semantic label.

12. The system of claim 11, wherein the received feature data further comprises at least one of:

payment-related information, device sensor data, external data, calendar data, communication data, or personal identification credentials data.

13. The system of claim 11, wherein at least some of the features of the visit are based upon a sequence of visits.

14. The system of claim 11, wherein at least some of the features of the visit are based upon crowd label-related features.

15. The system of claim 11 wherein the semantic label classifier includes or is coupled to user preference information or user exception information, or both user preference information and user exception information.

16. The system of claim 11 wherein the semantic label classifier comprises a boosted decision tree classifier.

17. One or more computer-readable storage devices having computer-executable instructions, which upon execution perform operations, comprising:

obtaining feature data associated with a plurality of location-related data corresponding to a plurality of visits associated with a user, the feature data including user demographics data and at least one of arrival day of week, arrival time of day, visit midpoint time of day, departure time of day, duration of visit, season of year, holiday data, and nearby business features;

classifying the location-related data corresponding to at least one visit into semantic label data for that visit based at least in part on the user demographic data and one or more other data from the feature data;

assigning a semantic label to the location-related data for that visit using the semantic label data; and

automatically performing an action that corresponds to the assigned semantic label, the action performed at a computing device associated with the user or at a computing device remote from the user.

18. The one or more computer-readable storage devices of claim 17 wherein the plurality of visits corresponds to a sequence of visits, and wherein classifying the location-related data comprises iteratively using probability data of an earlier classification iteration to determine at least one semantic label of a later classification iteration.

19. The one or more computer-readable storage devices of claim 18 wherein obtaining the feature data comprises determining at least one of:

sequence probability data, sequence periodicity data, sequence place type data, sequence place following data, sequence place type duration data or sequence place pairing data.