Automatic semantic labeling based on activity recognition

• US 9,565,521 B1
• Filed: 08/14/2015
• Issued: 02/07/2017
• Est. Priority Date: 08/14/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Pellet, Yannick, Hoque, Enamul, Srinivasan, Vijay, Mukherji, Abhishek
• Primary Examiner(s): Tran, Mong-Thuy
• Application Number:
  US14/826,332
• Publication Number:
  US 20170048660A1
• Time in Patent Office: 543 Days
• Field of Search: 455/457, 4554561-4565, 455/556.1
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0205 Location or geographical consideration
• CPC Class Codes: G06Q 50/06 Energy or water supply
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 1/73 Battery saving arrangements
• CPC Class Codes: H04W 24/00 Supervisory, monitoring or testing arrangements
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

A method and device for automatic semantic labeling of unlabeled places using activity recognition. A method includes determining at least one activity based on analyzing electronic device sensor data. Localization for the electronic device is performed to determine location for an unknown semantic place. An observed mapping between the at least one activity and the location for the unknown semantic place is determined. A typical mapping between the at least one activity and at least one semantic place is determined. Using the observed mapping and the typical mapping from one or more other electronic devices, the unknown semantic place is assigned with a semantic place label representing the at least one semantic place for identifying the unknown semantic place. A semantic place map is updated to include the semantic place label.

Claims

1. A method comprising:

recognizing an activity performed at a first place based on sensor data of an electronic device, wherein the first place comprises an unlabeled semantic place without an assigned semantic place label;

determining a location for the first place by performing localization for the electronic device;

determining an observed mapping between the activity and the location for the first place;

determining a typical mapping between the activity and a second place, wherein the second place comprises a labeled semantic place with an assigned semantic place label;

based on the observed mapping and the typical mapping, assigning the same sematic place label assigned to the labeled semantic place to the location for the first place; and

updating a semantic place map to include the semantic place label assigned to the location for the first place.

2. The method of claim 1, further comprising targeting at least one advertisement or incentive based message based on the location for the first place and the labeled semantic place.

3. The method of claim 1, further comprising:

determining a count of one or more participants participating in the activity at the location for the first place and the labeled semantic place.

4. The method of claim 1, further comprising:

providing messaging to a smart device based on the location for the first place and the labeled semantic place.

5. The method of claim 1, further comprising:

differentiating between two places with a same temporal occupancy pattern based on at least one of the activity and device identification information for the electronic device.

6. The method of claim 1, further comprising:

identifying one of a franchise name or a subcategory of the first place based on at least one of;

a unique activity pattern, device identification information for the electronic device, or a layout of the first place.

7. The method of claim 1, further comprising:

differentiating neighboring places based on the activity.

8. The method of claim 1, further comprising:

automatically updating a building layout with a sematic place label assigned to one of a room or an individual area; and

forwarding the updated semantic place map to an emergency responder.

9. The method of claim 1, further comprising:

determining a current location of the electronic device; and

reducing energy consumption based on the current location of the electronic device and the activity, wherein the reducing comprises at least one of;

turning off an appliance in a different location or adjusting a setting for a device.

10. The method of claim 1, further comprising:

reducing sampling rate for a sensor of the electronic device based on the activity and the location for the first place.

11. An apparatus comprising:

at least one processor; and

a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including;

recognizing an activity performed at a first place based on sensor data of an electronic device, wherein the first place comprises an unlabeled semantic place without an assigned semantic place label;

determining a location for the first place by performing localization for the electronic device;

determining an observed mapping between the activity and the location for the first place;

determining a typical mapping between the activity and a second place, wherein the second place comprises a labeled semantic place with an assigned semantic place label;

based on the observed mapping and the typical mapping, assigning the same sematic place label assigned to the labeled semantic place to the location for the first place; and

updating a semantic place map to include the semantic place label assigned to the location for the first place.

12. The apparatus of claim 11, wherein the operations further comprise:

receiving at least one advertisement or incentive based message based on the location for the first place and the labeled semantic place.

13. The apparatus of claim 11, wherein the operations further comprise:

determining a count of one or more participants in the activity at the location for the first place and the labeled semantic place.

14. The apparatus of claim 11, wherein the operations further comprise:

messaging to a smart device based on the determined location and the semantic place label.

15. The apparatus of claim 11, wherein the operations further comprise:

differentiating between two places with a same temporal occupancy pattern based on at least one of the activity and device identification information for the electronic device.

16. The apparatus of claim 11, wherein the operations further comprise:

identifying one of a franchise name or a subcategory of the first place based on at least one of;

a unique activity pattern, device identification information for the electronic device, or a layout of the first place.

17. The apparatus of claim 11, wherein the operations further comprise:

eliminating location noise by differentiating neighboring places based on the activity.

18. The apparatus of claim 11, wherein the operations further comprise:

updating a building layout with a sematic place label assigned to one of a room or an individual area; and

forwarding the updated semantic place map to an emergency responder.

19. The apparatus of claim 11, wherein the operations further comprise:

determining a current location of the electronic device; and

reducing energy consumption based on the current location of the electronic device and the activity, wherein the reducing comprises at least one of;

turning off an appliance in a different location or adjusting a setting for a device.

20. The apparatus of claim 11, wherein the operations further comprise:

reducing a sampling rate for a sensor of the electronic device based on the activity pattern and the location for the first place.

21. A non-transitory processor-readable medium that includes a program that when executed by a processor performs a method comprising:

recognizing an activity performed at a first place based on sensor data of an electronic device, wherein the first place comprises an unlabeled semantic place without an assigned semantic place label;

determining a location for the first place by performing localization for the electronic device;

determining an observed mapping between the activity and the location for the first place;

determining a typical mapping between the activity and a second place, wherein the second place comprises a labeled semantic place with an assigned semantic place label;

based on the observed mapping and the typical mapping, assigning the same sematic place label assigned to the labeled semantic place to the location for the first place; and

updating a semantic place map to include the semantic place label assigned to the location for the first place.

22. The non-transitory processor-readable medium of claim 21, further comprising targeting at least one advertisement or incentive based message based on the location for the first place and the labeled semantic place.

23. The non-transitory processor-readable medium of claim 21, further comprising:

determining a count of one or more participants participating in the activity at the location for the first place and the labeled semantic place.

24. The non-transitory processor-readable medium of claim 21, further comprising:

providing messaging to a smart device based on the location for the first place and the labeled semantic place.

25. The non-transitory processor-readable medium of claim 21, further comprising:

differentiating between two places with a same temporal occupancy pattern based on at least one of the activity and device identification information for the electronic device.

26. The non-transitory processor-readable medium of claim 21, further comprising:

identifying one of a franchise name or a subcategory of the first place based on at least one of;

a unique activity pattern, device identification information for the electronic device, or a layout of the first place.

27. The non-transitory processor-readable medium of claim 21, further comprising differentiating neighboring places based on the activity.

28. The non-transitory processor-readable medium of claim 21, further comprising:

automatically updating a building layout with a sematic place label assigned to one of a room or an individual area; and

forwarding the updated semantic place map to an emergency responder.

29. The non-transitory processor-readable medium of claim 21, further comprising:

determining a current location of the electronic device; and

reducing energy consumption based on the current location of the electronic device and the activity, wherein the reducing comprises at least one of;

turning off an appliance in a different location or adjusting a setting for a device.

30. The non-transitory processor-readable medium of claim 21, further comprising:

reducing sampling rate for a sensor of the electronic device based on the activity and the location for the first place.