Identifying status based on heterogeneous sensors

• US 9,497,594 B2
• Filed: 04/10/2014
• Issued: 11/15/2016
• Est. Priority Date: 04/18/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Xie, Xing, Zhu, Yin, Arase, Yuki
• Primary Examiner(s): MEHRA, INDER P
• Application Number:
  US14/250,297
• Publication Number:
  US 20140221020A1
• Time in Patent Office: 950 Days
• Field of Search: 455/404.1, 455/404.2, 455/411, 455/414.1, 455/418, 455/456.1, 455/456.3, 455/557, 455/420, 455/410, 455/566, 455/550.1, 455/41.3, 455/68, 340/10.5, 340/501, 340/506, 348/143, 348/146, 709/202, 709/232, 704/226, 704/240, 370/338, 370/328, 370/310, 370/401, 370/254, 370/310.2, 370/312, 370/402, 370/311, 370/331, 370/339, 370/347, 370/35, 370/2, 370/392, 370/400, 370/242, 370/448, 705/14.49, 705/14.58, 705/14.66, 705/28, 705/318, 705/325, 705/7.34
• US Class Current: 1/1
• CPC Class Codes: G01C 22/00 Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 4/025 using location based information parameters
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/027 using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

Techniques for determining a status of a user are described. A mobile device equipped with sensors may collect sensor data pertaining to transportation modes of the user, tracking locations of the user, identifying environmental noise levels surrounding the user, or speech being spoken in proximity to the user. Features of the collected sensor readings are then used to infer activities the user may be performing. Based at least in part on the multiple inferred activities, a status of the user is determined.

Claims

1. A system comprising:

a memory;

a processor coupled to the memory;

a first sensor configured to collect first acceleration data associated with a first time and second acceleration data associated with a second time;

a second sensor configured to collect first location data associated with the first time and second location data associated with the second time;

a status module stored in the memory and executable by the processor to;

determine, based at least in part on the first acceleration data and the first location data, a first activity status of a user of the system;

determine, based at least in part on the second acceleration data and the second location data, a second activity status of the user; and

outputting a recommendation for presentation to the user based at least in part on a difference between the first activity status and the second activity status.

2. The system of claim 1, wherein the first activity status indicates that the user is stationary, moving, taking stairs, riding in an escalator, riding in an elevator, or riding in a transportation vehicle.

3. The system of claim 1, wherein the status module is further configured to integrate a context of the user for applications, services, and other computing devices by sending an alert or a notification of the first activity status of the user.

4. The system of claim 1, further comprising another module to collect additional sensor data to augment transportation modes of the user or to augment locations of the user.

5. The system of claim 1, wherein collecting the first location data and the second location data comprises:

detecting at least a signal from one of;

a Wi-Fi access point, a Global System for Mobile Communications (GSM), a Personal Area Network (PAN), a Global Positioning System (GPS), or a broadcast radio signal.

6. The system of claim 1, further comprising a third sensor configured to record an environmental sound surrounding the user including a background noise level.

7. The system of claim 1, further comprising a third sensor configured to record speech being spoken in proximity to the user, the recording comprising:

detecting a speech from a speaker'"'"'s voice; and

extracting acoustic features from each voice segment of the speaker'"'"'s voice.

8. The system of claim 1, wherein the recommendation is to be provided via a text message or an email.

9. A method implemented at least in part by one or more processors, the method comprising:

collecting, from a first sensor, first acceleration data associated with a first time and second acceleration data associated with a second time;

collecting, from a second sensor, first location data associated with the first time and second location data associated with the second time; and

providing instructions to a status module stored in memory and executable by the one or more processors to;

determine, based at least in part on the first acceleration data and the first location data, a first activity status of a user;

determine, based at least in part on the second acceleration data and the second location data, a second activity status of the user; and

providing a recommendation based at least in part on a difference between the first activity status and the second activity status.

10. The method of claim 9, further comprising identifying whether the user is stationary, moving, taking stairs, riding in an escalator, riding in an elevator, or riding in a transportation vehicle.

11. The method of claim 9, wherein collecting the first location data and the second location data comprises:

detecting at least a signal from one of;

a Wi-Fi access point, a Global System for Mobile Communications (GSM), a Personal Area Network (PAN), a Global Positioning System (GPS), or a broadcast radio signal.

12. The method of claim 9, further comprising recording an environmental sound surrounding the user including a background noise level.

13. The method of claim 9, further comprising recording speech being spoken in proximity to the user, the recording comprising:

detecting a speech from a speaker'"'"'s voice; and

extracting acoustic features from each voice segment of the speaker'"'"'s voice.

14. The method of claim 9, wherein the first activity status of the user comprises at least one of working in an office, participating in a meeting, moving in a building, eating a meal, or attending a seminar.

15. The method of claim 9, further comprising augmenting the first acceleration data, the second acceleration data, the first location data, or the second location data with data collected from at least one of:

a compass, a magnetometer, an ambient light sensor, a camera, a microphone, a sound level meter, an amplitude modulated (AM) radio frequency module, or a frequency modulated (FM) radio frequency module.

16. The method of claim 9, further comprising providing, to other devices, applications, or services, the first or second activity status of the user as a context for the other devices, applications, or services.

17. A system comprising:

memory;

one or more processors communicatively coupled to the memory;

a first sensor configured to collect first environmental sound data associated with a first time and second environmental sound data associated with a second time;

a second sensor configured to collect at least one of;

first location data associated with the first time and second location data associated with the second time;

or

first acceleration data associated with a first time and second acceleration data associated with a second time; and

a status module stored in the memory and executable by the one or more processors to;

determine, based at least in part on the first environmental sound data collected from the first sensor and the at least one of the first location data or the first acceleration data collected from the second sensor, a first activity status of a user of the system;

determine, based at least in part on the second environmental sound data collected from the first sensor and the at least one of the second location data or the second acceleration data collected from the second sensor, a second activity status of the user; and

provide a recommendation to the user based at least in part on a difference between the first activity status and the second activity status.

18. The system of claim 17, wherein the recommendation is to be provided via a text message or an email.

19. The system of claim 17, wherein the recommendation to be provided comprises a suggestion to the user to perform a past activity previously recorded by the status module.

20. The system of claim 17, wherein the status module is further executable to provide, to other devices, applications, or services, the first activity status or the second activity status of the user as a context for the other devices, applications, or services.