Identifying Status Based on Heterogeneous Sensors

• US 20120264446A1
• Filed: 04/18/2011
• Published: 10/18/2012
• Est. Priority Date: 04/18/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Xie, Xing, Zhu, Yin, Arase, Yuki
• Granted Patent: US 8,718,672 B2
• Time in Patent Office: Days
• US Class Current: 455/456.1
• CPC Class Codes: G01C 22/00 Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 4/025 using location based information parameters
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/027 using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

Techniques for determining a status of a user are described. A mobile device equipped with sensors may collect sensor data pertaining to transportation modes of the user, tracking locations of the user, identifying environmental noise levels surrounding the user, or speech being spoken in proximity to the user. Features of the collected sensor readings are then used to infer activities the user may be performing. Based at least in part on the multiple inferred activities, a status of the user is determined.

Claims

1. A method comprising:

collecting sensor data from sensors on a mobile device;

inferring from the sensor data a transportation mode, a location, an environmental condition surrounding a mobile device, and/or speech heard in proximity to the mobile device based on at least;

an accelerometer or a barometer to determine the transportation mode of a user of the mobile device;

a detector to track the location of the user of the mobile device;

a microphone to record the environmental condition surrounding the mobile device or the speech being spoken in proximity to the mobile device; and

designating a status of the user based at least in part on analysis of the transportation mode, the location, the environmental condition, or the speech.

2. The method of claim 1, wherein determining the transportation mode comprises:

identifying whether the user is stationary or moving; and

identifying whether the user is changing elevation when a barometric pressure change occurs at a predetermined rate during a time interval.

3. The method of claim 1, wherein determining the transportation mode comprises identifying whether the user is stationary, moving, taking stairs, riding in an escalator, riding in an elevator, or riding in transportation vehicle.

4. The method of claim 1, wherein tracking the location of the user comprises:

detecting at least a signal from one of;

a Wi-Fi access point, a Global System for Mobile Communications (GSM), a Personal Area Network (PAN), a Global Positioning System (GPS), or a broadcast radio signal;

or

detecting a change in a temperature reading from a temperature sensor for a predetermined time interval.

5. The method of claim 1, wherein recording the environmental condition surrounding the mobile device comprises automatically recording a background noise level.

6. The method of claim 1, wherein recording the speech being spoken in proximity to the mobile device comprises:

detecting a speech from a speaker'"'"'s voice; and

extracting acoustic features from each voice segment of the speaker'"'"'s voice.

7. The method of claim 1, wherein the status of the user comprises at least one of working in an office, participating in a meeting, moving in a building, eating a meal, or attending a seminar.

8. The method of claim 1, further comprising augmenting the transportation mode of the user or the location of the user with data collected from at least one of:

a compass, a magnetometer, an ambient light sensor, a camera, a microphone, a sound level meter, an amplitude modulated (AM) radio frequency module, or a frequency modulated (FM) radio frequency module.

9. The method of claim 1, further comprising:

monitoring a behavior of the user that has occurred over a time period;

recording a modification of the monitored behavior;

providing a recommendation to the user based at least in part on a difference between the modified behavior and the monitored behavior.

10. The method of claim 1, further comprising integrating the status of the user as a context for other devices, applications, or services.

11. One or more computer-readable media encoded with instructions that, when executed by a processor of a mobile device, perform operations comprising:

collecting sensor data including;

a magnitude of accelerometer data or a barometric pressure of a barometer to identify movement of a user of the mobile device;

a signal from a detector to track locations of the user of the mobile device;

a recording from a microphone to identify environmental conditions surrounding the user or to detect speech being spoken in proximity to the user; and

determining a status of the user based on the movement, the locations, the environmental conditions, or the speech.

12. The one or more computer-readable media of claim 11, wherein:

the magnitude is a change in velocity with respect to an inertial space to determine whether the user of the mobile device is moving or is stationary; and

the barometric pressure represents a change in altitude to determine whether the user is changing floor levels, taking stairs, riding an escalator, or riding an elevator.

13. The one or more computer-readable media of claim 11, wherein tracking locations of the user comprises detecting at least a signal from one of:

a Wi-Fi access point, a Global System for Mobile Communications (GSM), a Personal Area Network (PAN), or a Global Positioning System (GPS).

14. The one or more computer-readable media of claim 11, wherein:

identifying the environmental conditions surrounding the user comprises recording an ambient noise level in a background; and

detecting the speech being spoken in proximity to the user comprises extracting acoustic features from voice segments of a speaker'"'"'s voice.

15. The one or more computer-readable media of claim 11, the operations further comprising:

recording directional information from a compass;

associating the directional information with the movement or the locations; and

providing recommendations based on the status of the user.

16. A system comprising:

a memory;

a processor coupled to the memory;

a plurality of modules stored in the memory and executable on the processor, the plurality of modules comprising;

a status application module configured to collect sensor data on a mobile device;

an accelerometer module or a barometer module to identify transportation modes of a user of the mobile device;

a Wi-Fi module, a Global System for Mobile Communications (GSM) module, a Personal Area Network (PAN) module, or a Global Positioning System (GPS) module to track locations of the user of the mobile device;

a microphone module to record environmental conditions surrounding the user of the mobile device, and to record speech being spoken in proximity to the user of the mobile device.

17. The system of claim 16, wherein the transportation modes include the user is stationary, moving, taking stairs, riding in an escalator, riding in an elevator, or riding in a transportation vehicle.

18. The system of claim 16, wherein the status application module is further configured to determine a status of the user based on the sensor data, the status of the user including at least one of working in an office, participating in a meeting, moving in a building, eating a meal, or attending a seminar.

19. The system of claim 18, wherein the status application module is further configured to integrate a context of the user for applications, services, and other computing devices by sending an alert or a notification of the status of the user being unavailable when the user is participating in a meeting or attending a seminar.

20. The system of claim 16, further comprising a compass module, a camera module, or an ambient light module to collect sensor data to augment the transportation modes of the user or to augment tracking locations of the user.