User activity tracking system and device

• US 9,125,015 B2
• Filed: 06/28/2013
• Issued: 09/01/2015
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Pennanen, Juho, Kyrl, Aapo
• Primary Examiner(s): Eisner, Ronald
• Application Number:
  US13/930,347
• Publication Number:
  US 20150004998A1
• Time in Patent Office: 795 Days
• Field of Search: 4554141-4143, 455418-420, 455/404.1, 455/404.2, 4554561-457, 340/539.13, 340988-996
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2291 User-Defined Types; Storage management thereof
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/027 using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

The present disclosure provides a system for tracking and recording movements of a mobile communication device that includes one or more movement sensors. The mobile communication device communicates sensor signals to the system, wherein the sensor signals are indicative of motion associated with activities to which the mobile communication device is exposed by its user. The sensor signals are pre-classified to generate the intermediate data, where the intermediate data is processed in one or more processors to generate indications of likely activities associated with the sensor signals, and an aggregate of the indications is computed to provide an analysis of activities associated with the sensor signals. The information indicating most likely activity types is then sent to the mobile communication device.

Claims

1. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to;

receive from a mobile communication device one or more sensor signals indicative of motion associated with activities of a user of the mobile communication device;

determine from a plurality of activity categories a particular activity category that corresponds to a portion of the sensor signals, wherein each of the activity categories comprises one or more activity types and is characterized by motion that the activity types in the activity category have in common;

determine from among the activity types in the particular activity category one or more activity types of the user by;

analyzing the sensor signals against each of the activity types in the particular activity category; and

calculating a probability of each of the activity types of the user based on the analysis; and

send the activity types to the mobile communication device.

2. The system of claim 1, wherein the one or more processors are configured to process the sensor signals substantially in parallel, wherein the processors are mutually specialized in identifying characteristics of the sensor signals corresponding to one or more activity types.

3. The system of claim 1, wherein determining the activity types comprises generating a temporal log of activity types experienced by the mobile communication device.

4. The system of claim 1, wherein mobile communication device comprises a personal computer, a portable media device, a smart phone, a wrist-worn phone, a phablet, a mobile telephone, or a tablet computer.

5. The system of claim 1, wherein the mobile communication device comprises one or more sensors comprising a gyroscopic angular sensor, an accelerometer, a GPS position sensor, cellular positioning sensor, a magnetometer, a microphone, a camera, or a temperature sensor.

6. The system of claim 1, wherein the analysis of the sensor signals comprises:

a supervised or semisupervised classification analysis;

or

a heuristic analysis.

7. The system of claim 6, wherein the supervised or semisupervised classification analysis comprises classification algorithms that use as input the amplitudes of the frequency components of the information included in the sensor signals, and the output of the classification algorithms are estimated probabilities of different activity types based on the sensor signals.

8. The system of claim 1, wherein sending the activity types comprises sending analyzed activity results to the mobile computing device for display on a graphical user interface of a software application of the mobile computing device, wherein the analyzed activity results comprise a timeline comprising a plurality of mutually different symbols representing different analyzed activity types.

9. A method comprising

by a computing device, receiving from a mobile communication device one or more sensor signals indicative of motion associated with activities of a user of the mobile communication device;

by a computing device, determining from a plurality of activity categories a particular activity category that corresponds to a portion of the sensor signals, wherein each of the activity categories comprises one or more activity types and is characterized by motion that the activity types in the activity category have in common;

by a computing device, determining from among the activity types in the particular activity category one or more activity types of the user by;

analyzing the sensor signals against each of the activity types in the particular activity category; and

calculating a probability of each of the activity types of the user based on the analysis; and

by a computing device, sending the activity types to the mobile communication device.

10. The method of claim 9, wherein the sensor signals are analyzed substantially in parallel by one or more processors of the computing device, the processors being mutually specialized in identifying characteristics of the sensor signals corresponding to one or more activity types.

11. The method of claim 9, wherein determining the activity types comprises generating a temporal log of activity types experienced by the mobile communication device.

12. The method of claim 9, wherein the mobile communication device comprises a personal computer, a portable media device, a smart phone, a wrist-worn phone, a phablet, a mobile telephone, or a tablet computer.

13. The method of claim 9, wherein mobile communication device comprises one or more sensors comprising a gyroscopic angular sensor, an accelerometer, a GPS position sensor, cellular positioning sensor, a magnetometer, a microphone, a camera, or a temperature sensor.

14. The method of claim 9, wherein the analysis of the sensor signals comprises:

a supervised or semisupervised classification analysis;

or

a heuristic analysis.

15. The method of claim 14, wherein the supervised or semisupervised classification analysis comprises classification algorithms that use as input the amplitudes of the frequency components of the information included in the sensor signals, and the output of the classification algorithms are estimated probabilities of different activity types based on the sensor signals.

16. The method of claim 9, wherein sending the activity types comprises sending analyzed activity results to the mobile computing device for display on a graphical user interface of a software application of the mobile computing device, wherein the analyzed activity results comprise a timeline comprising a plurality of mutually different symbols representing different analyzed activity types.

17. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive from a mobile communication device one or more sensor signals indicative of motion associated with activities of a user of the mobile communication device;

determine from a plurality of activity categories a particular activity category that corresponds to a portion of the sensor signals, wherein each of the activity categories comprises one or more activity types and is characterized by motion that the activity types in the activity category have in common;

determine from among the activity types in the particular activity category one or more activity types of the user by;

analyzing the sensor signals against each of the activity types in the particular activity category;

calculating a probability of each of the activity types of the user based on the analysis; and

send the activity types to the mobile communication device.

18. The media of claim 17, wherein determining the activity types comprises generating a temporal log of activity types experienced by the mobile communication device.

20. The media of claim 17, wherein the sensor signals are analyzed substantially in parallel by one or more processors executing the software, the processors being mutually specialized in identifying characteristics of the sensor signals corresponding to one or more activity types.

21. The media of claim 17, wherein the mobile communication device comprises a personal computer, a portable media device, a smart phone, a wrist-worn phone, a phablet, a mobile telephone, or a tablet computer.

22. The media of claim 17, wherein the mobile communication device comprises one or more sensors comprising a gyroscopic angular sensor, an accelerometer, a GPS position sensor, cellular positioning sensor, a magnetometer, a microphone, a camera, or a temperature sensor.

23. The media of claim 17, wherein the analysis of the sensor signals comprises:

a supervised or semisupervised classification analysis;

or

a heuristic analysis.

24. The media of claim 23, wherein the supervised or semisupervised classification analysis comprises classification algorithms that use as input the amplitudes of the frequency components of the information included in the sensor signals, and the output of the classification algorithms are estimated probabilities of different activity types based on the sensor signals.

25. The media of claim 17, wherein sending the activity types comprises sending analyzed activity results to the mobile computing device for display on a graphical user interface of a software application of the mobile computing device, wherein the analyzed activity results comprise a timeline comprising a plurality of mutually different symbols representing different analyzed activity types.

19. A device comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to;

send to a server computing system one or more sensor signals of the device, wherein the sensor signals comprise indications of motion associated with activities of a user of the device; and

receive from the server computing system one or more activity types based on;

determining from a plurality of activity categories a particular activity category that corresponds to a portion of the sensor signals, wherein each of the activity categories comprises one or more activity types and is characterized by motion that the activity types in the activity category have in common; and

determining the activity types of the user in the activity category by;

analyzing the sensor signals against each of the activity types in the particular activity category; and

calculating a probability of each of the activity types of the user based on the analysis.

26. The device of claim 19, wherein the sensor signals are analyzed substantially in parallel by one or more processors of the server computing system, the processors being mutually specialized in identifying characteristics of the sensor signals corresponding to one or more activity types.

27. The device of claim 19, wherein determining the activity types comprises generating a temporal log of activity types experienced by the device.

28. The device of claim 19, wherein the device comprises a personal computer, a portable media device, a smart phone, a wrist-worn phone, a phablet, a mobile telephone, or a tablet computer.

29. The device of claim 19, wherein the device comprises one or more sensors comprising a gyroscopic angular sensor, an accelerometer, a GPS position sensor, cellular positioning sensor, a magnetometer, a microphone, a camera, or a temperature sensor.

30. The device of claim 19, wherein the analysis of the sensor signals comprises:

a supervised or semisupervised classification analysis;

or

a heuristic analysis.

31. The device of claim 30, wherein the supervised or semisupervised classification analysis comprises classification algorithms that use as input the amplitudes of the frequency components of the information included in the sensor signals, and the output of the classification algorithms are estimated probabilities of different activity types based on the sensor signals.

32. The device of claim 19, wherein receiving the activity types comprises receiving analyzed activity results from the server computing system for display on a graphical user interface of a software application of the device, wherein the analyzed activity results comprise a timeline comprising a plurality of mutually different symbols representing different analyzed activity types.

33. A method comprising:

by a client computing device, sending to a server computing system one or more sensor signals of the client computing device, wherein the sensor signals comprise indications of motion associated with activities of a user of the client computing device; and

by the client computing device, receiving from the server computing system one or more activity types based on;

determining from a plurality of activity categories a particular activity category that corresponds to a portion of the sensor signals, wherein each of the activity categories comprises one or more activity types and is characterized by motion that the activity types in the activity category have in common; and

determining the activity types of the user in the activity category by;

analyzing the sensor signals against each of the activity types in the particular activity category; and

calculating a probability of each of the activity types of the user based on the analysis.

34. The method of claim 33, wherein the sensor signals are analyzed substantially in parallel by one or more processors of the server computing system, the processors being mutually specialized in identifying characteristics of the sensor signals corresponding to one or more activity types.

35. The method of claim 33, wherein determining the activity types comprises generating a temporal log of activity types experienced by the client computing device.

36. The method of claim 33, wherein the client computing device comprises a personal computer, a portable media device, a smart phone, a wrist-worn phone, a phablet, a mobile telephone, or a tablet computer.

37. The method of claim 33, wherein the client computing device comprises one or more sensors comprising a gyroscopic angular sensor, an accelerometer, a GPS position sensor, cellular positioning sensor, a magnetometer, a microphone, a camera, or a temperature sensor.

38. The method of claim 33, wherein the analysis of the sensor signals comprises:

a supervised or semisupervised classification analysis;

or

a heuristic analysis.

39. The method of claim 38, wherein the supervised or semisupervised classification analysis comprises classification algorithms that use as input the amplitudes of the frequency components of the information included in the sensor signals, and the output of the classification algorithms are estimated probabilities of different activity types based on the sensor signals.

40. The method of claim 33, wherein receiving the activity types comprises receiving analyzed activity results from the server computing system for display on a graphical user interface of a software application of the client computing device, wherein the analyzed activity results comprise a timeline comprising a plurality of mutually different symbols representing different analyzed activity types.

41. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

send to a server computing system one or more sensor signals, wherein the sensor signals comprise indications of motion associated with activities of a user; and

receive from the server computing system one or more activity types based on;

determining from a plurality of activity categories a particular activity category that corresponds to a portion of the sensor signals, wherein each of the activity categories comprises one or more activity types and is characterized by motion that the activity types in the activity category have in common; and

determining the activity types of the user in the activity category by;

analyzing the sensor signals against each of the activity types in the particular activity category; and

calculating a probability of each of the activity types of the user based on the analysis.

42. The media of claim 41, wherein the sensor signals are analyzed substantially in parallel by one or more processors of the server computing system, the processors being mutually specialized in identifying characteristics of the sensor signals corresponding to one or more activity types.

43. The media of claim 41, wherein determining the activity types comprises generating a temporal log of activity types experienced by the user.

44. The media of claim 41, wherein the sensor signals are generated by a device comprising a personal computer, a portable media device, a smart phone, a wrist-worn phone, a phablet, a mobile telephone, or a tablet computer.

45. The media of claim 41, wherein the sensor signals are generated by a device comprising one or more sensors comprising a gyroscopic angular sensor, an accelerometer, a GPS position sensor, cellular positioning sensor, a magnetometer, a microphone, a camera, or a temperature sensor.

46. The media of claim 41, wherein the analysis of the sensor signals comprises:

a supervised or semisupervised classification analysis;

or

a heuristic analysis.

47. The media of claim 46, wherein the supervised or semisupervised classification analysis comprises classification algorithms that use as input the amplitudes of the frequency components of the information included in the sensor signals, and the output of the classification algorithms are estimated probabilities of different activity types based on the sensor signals.

48. The media of claim 41, wherein receiving the activity types comprises receiving analyzed activity results from the server computing system for display on a graphical user interface, wherein the analyzed activity results comprise a timeline comprising a plurality of mutually different symbols representing different analyzed activity types.