Identifying status based on heterogeneous sensors

• US 8,718,672 B2
• Filed: 04/18/2011
• Issued: 05/06/2014
• Est. Priority Date: 04/18/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Xie, Xing, Zhu, Yin, Arase, Yuki
• Primary Examiner(s): MEHRA, INDER P
• Application Number:
  US13/088,755
• Publication Number:
  US 20120264446A1
• Time in Patent Office: 1,114 Days
• Field of Search: 455/404.1, 455/404.2, 455/411, 455/414.1, 455/418, 455/456.1, 455/456.3, 340/10.5, 340/501, 340/506, 348/143, 348/146, 709/202, 709/232, 704/226, 704/240
• US Class Current: 455/456.1
• CPC Class Codes: G01C 22/00 Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 4/025 using location based information parameters
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/027 using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

Techniques for determining a status of a user are described. A mobile device equipped with sensors may collect sensor data pertaining to transportation modes of the user, tracking locations of the user, identifying environmental noise levels surrounding the user, or speech being spoken in proximity to the user. Features of the collected sensor readings are then used to infer activities the user may be performing. Based at least in part on the multiple inferred activities, a status of the user is determined.

Claims

1. A method comprising:

collecting sensor data from sensors on a mobile device;

inferring from the sensor data a transportation mode, a location and an environmental condition surrounding the mobile device;

designating a status of a user of the mobile device based at least in part on analysis of the transportation mode, the location and the environmental condition;

monitoring, by a status application of the mobile device, an activity of the user;

detecting, by the status application, a modification of the monitored activity; and

providing, by the status application, a recommendation to the user based at least in part on a difference between the modified activity and the monitored activity.

2. The method of claim 1, further comprising determining the transportation mode using a barometer, the determining comprising:

identifying whether the user is stationary or moving; and

identifying whether the user is changing elevation when a barometric pressure change occurs at a predetermined rate during a time interval.

3. The method of claim 1, further comprising determining the transportation mode to identify whether the user is stationary, moving, taking stairs, riding in an escalator, riding in an elevator, or riding in transportation vehicle.

4. The method of claim 1, further comprising tracking the location of the user, the tracking comprising:

detecting at least a signal from one of;

a Wi-Fi access point, a Global System for Mobile Communications (GSM), a Personal Area Network (PAN), a Global Positioning System (GPS), or a broadcast radio signal;

or

detecting a change in a temperature reading from a temperature sensor for a predetermined time interval.

5. The method of claim 1, further comprising recording the environmental condition surrounding the mobile device, the recording comprising automatically recording a background noise level.

6. The method of claim 1, further comprising recording speech being spoken in proximity to the mobile device, the recording comprising:

detecting a speech from a speaker'"'"'s voice; and

extracting acoustic features from each voice segment of the speaker'"'"'s voice.

7. The method of claim 1, wherein the status of the user comprises at least one of working in an office, participating in a meeting, moving in a building, eating a meal, or attending a seminar.

8. The method of claim 1, further comprising augmenting the transportation mode of the user or the location of the user with data collected from at least one of:

a compass, a magnetometer, an ambient light sensor, a camera, a microphone, a sound level meter, an amplitude modulated (AM) radio frequency module, or a frequency modulated (FM) radio frequency module.

9. The method of claim 1, further comprising integrating the status of the user as a context for other devices, applications, or services.

10. A mobile device comprising:

a memory;

a processor coupled to the memory;

a plurality of modules stored in the memory and executable on the processor, the plurality of modules comprising;

a status application module configured to;

collect sensor data on the mobile device,

monitor an activity of a user of the mobile device,

detect a modification of the monitored activity, and

provide a recommendation to the user of the mobile device based at least in part on a difference between the modified activity and the monitored activity;

an accelerometer module or a barometer module to identify transportation modes of the user of the mobile device;

a Wi-Fi module, a Global System for Mobile Communications (GSM) module, a Personal Area Network (PAN) module, and/or a Global Positioning System (GPS) module to track locations of the user of the mobile device;

a microphone module to record environmental conditions surrounding the user of the mobile device, and to record speech being spoken in proximity to the user of the mobile device.

11. The mobile device of claim 10, wherein the transportation modes include the user is stationary, moving, taking stairs, riding in an escalator, riding in an elevator, or riding in a transportation vehicle.

12. The mobile device of claim 10, wherein the status application module is further configured to determine a status of the user based on the sensor data, the status of the user including at least one of working in an office, participating in a meeting, moving in a building, eating a meal, or attending a seminar.

13. The mobile device of claim 12, wherein the status application module is further configured to integrate a context of the user for applications, services, and other computing devices by sending an alert or a notification of the status of the user being unavailable when the user is participating in a meeting or attending a seminar.

14. The mobile device of claim 10, further comprising a compass module, a camera module, or an ambient light module to collect sensor data to augment the transportation modes of the user or to augment tracking locations of the user.

15. One or more computer storage media encoded with instructions that, when executed by a processor of a mobile device, perform operations comprising:

receiving an explicit consent from a user of a mobile device for sensor data tracking;

collecting, by a status application included in the mobile device of the user, sensor data from sensors of the mobile device;

inferring, by the status application, an activity of the user of the mobile device based on the collected sensor data;

monitoring, by the status application, the activity of the user of the mobile device;

recording, by the status application, a modification of the monitored activity;

providing, by the status application, a recommendation to the user of the mobile device based at least in part on a difference between the modified activity and the monitored activity.

16. The one or more computer storage media of claim 15, wherein:

the sensor data comprises a magnitude of accelerometer data or a barometric pressure of a barometer to identify movement of the user of the mobile device;

the magnitude is a change in velocity with respect to an inertial space to determine whether the user of the mobile device is moving or is stationary; and

the barometric pressure represents a change in altitude to determine whether the user is changing floor levels, taking stairs, riding an escalator, or riding an elevator.

17. The one or more computer storage media of claim 15, wherein the sensor data comprises a signal from a detector to track locations of the user, the signal comprising one of:

a Wi-Fi access point, a Global System for Mobile Communications (GSM), a Personal Area Network (PAN), or a Global Positioning System (GPS).

18. The one or more computer storage media of claim 15, wherein:

the sensor data comprises a recording from a microphone to identify environmental conditions surrounding the user or to detect speech being spoken in proximity to the user;

identifying the environmental conditions surrounding the user comprises recording an ambient noise level in a background; and

detecting the speech being spoken in proximity to the user comprises extracting acoustic features from voice segments of a speaker'"'"'s voice.

19. One or more computer storage media of claim 15, wherein providing the recommendation comprises providing a suggestion to the user to perform a past activity previously recorded by the status application via a text message or an email.