Physical activity inference from environmental metrics

• US 9,700,240 B2
• Filed: 12/14/2012
• Issued: 07/11/2017
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Brush, Alice Jane, Morris, Daniel, Letchner, Julia Maureen, Saponas, Scott, Kansal, Aman
• Primary Examiner(s): MULLEN, THOMAS J
• Application Number:
  US13/714,547
• Publication Number:
  US 20140167973A1
• Time in Patent Office: 1,670 Days
• Field of Search: 3405731, 3405734, 34053911-53913, 34053926, 34087016, 702127, 702141, 700 91
• CPC Class Codes: A61B 2503/12 Healthy persons not otherwise provided for, e.g. subjects of a marketing survey
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 2560/0247 for compensation or correction of the measured physiological value
• CPC Class Codes: A61B 2562/0204 Acoustic sensors
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 2562/0223 Magnetic field sensors
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/6801 specially adapted to be attached to or worn on the body surface
• CPC Class Codes: A61B 5/7475 User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0243 Comparative campaigns
• CPC Class Codes: G06Q 30/0244 Optimization
• CPC Class Codes: G06Q 30/0245 Surveys
• CPC Class Codes: G06Q 30/0254 based on statistics
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06Q 30/0613 Third-party assisted
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Abstract

Portable devices include environmental sensors that generate metrics about the environment (e.g., accelerometers detecting impulses and vibration, and GPS receivers detecting position and velocity). Such devices often use environmental metrics to extract user input directed at the device by the user, and status information about the device and the environment. Presented herein are techniques for using environmental metrics to infer physical activities performed by the user while attached to the device. For example, jogging may be inferred from regular, strong impulses and typical jogging speed; walking may be inferred from regular, weak impulses and typical walking speed; and riding in a vehicle may be inferred from low-level vibrations and high speed (optionally identifying the type of vehicle ridden by the user). Based on these inferences, the device may automatically present applications and/or or adjust user interfaces suitable for the user'"'"'s physical activity, rather than responsive to user input.

Claims

1. A method for presenting a user interface for an application on an electronic device, comprising:

detecting, using at least one environmental sensor associated with the electronic device, at least one environmental metric associated with a physical activity performed by a user wearing the electronic device; and

selecting a user interface mode from at least two interface modes of the user interface, based at least in part, on the at least one environmental metric associated with the physical activity, wherein the selected user interface mode includes one or more user interface elements that comprise at least one of;

a user interface information element that presents information; and

a user interface control element that enables control of the application;

identifying, among at least two user interface presentations of the user interface element, a selected user interface presentation of the user interface element that is associated with the selected user interface mode; and

presenting the user interface element according to the selected user interface presentation.

2. The method of claim 1, wherein the environmental metric is selected from an environmental metric set comprising:

a geolocation metric;

a linear velocity metric;

an angular velocity metric;

an acceleration metric;

an electronic device orientation metric;

a light level metric;

a sound level metric;

a magnetic field metric; and

a wireless communication signal metric.

3. The method of claim 1, wherein the physical activity is selected from a physical activity set comprising:

a walking physical activity;

a standing physical activity;

a sitting physical activity;

a jogging physical activity;

a swimming physical activity; and

a bicycling physical activity.

4. The method of claim 1, wherein:

the physical activity further comprises a vehicle riding physical activity and when the vehicle riding activity is identified, the method further comprises;

results in the environmental metric.

5. The method of claim 1, wherein detecting the at least one environmental metric associated with the physical activity further comprises:

sending at least one environmental metric to a physical activity identification service; and

receiving, from the physical activity identification service, an identification of the physical activity according to the at least one environmental metric.

6. The method of claim 1, wherein:

the electronic device further comprises a physical activity profile associated with a physical activity that is identifiable by at least one profile environmental metric that is detectable by the at least one environmental sensor of the electronic device while the physical activity is performed; and

the detecting comprising;

determining whether the at least one environmental metric detected by the at least one environmental sensor matches the at least one profile environmental metric of the physical activity profiles to identify a selected physical activity profile; and

responsive to determining that the at least one environmental metric detected by the at least one environmental sensor matches the at least one profile environmental metric of the physical activity profile, selecting the physical activity associated with the at least one profile environmental metric as a detected physical activity.

7. The method of claim 1, wherein at least one of the user interface elements further comprises:

a first user interface presentation of the user interface element that communicates with the user through a first user interface modality corresponding to a first user interface mode; and

a second user interface presentation of the user interface element that communicates with the user through a second user interface modality corresponding to a second user interface mode.

8. A system that causes a wearable device to present an application in at least two user interface modes, with each user interface mode corresponding to a physical activity performed by a user wearing the wearable device, wherein the wearable device comprises an environmental sensor, the system comprising:

a physical activity receiver that receives, from the environmental sensor, at least one environmental metric that does not include user input;

a physical activity identifier that;

identifies, using the at least one environmental metric, a performance of the physical activity; and

identifies, among the at least two user interface modes, a selected user interface mode according to the performance of the physical activity, wherein the selected user interface mode includes a user interface element that comprises one or more of a user interface information element that presents information and a user interface control element that enables control of the application; and

a user interface presenter that;

identifies, among at least two user interface presentations of the user interface element, a selected user interface element presentation that matches the selected user interface mode; and

presents the user interface element according to the selected user interface presentation.

9. The system of claim 8, further comprising a physical activity reporter that:

responsive to receiving from at least one application a request to receive reports of physical activity, stores the request; and

responsive to identifying a physical activity that is associated with a request, reports the physical activity to the application associated with the request.

10. The system of claim 8, further comprising an environmental sensor controller that:

enables the environmental sensor to detect the at least one environmental metric responsive to receiving a request to identify the physical activity; and

disables the environmental sensor after reporting the physical activity.

11. The system of claim 8, further comprising:

a physical activity set comprising at least two physical activities; and

classifier logic that;

identifies, among the at least two physical activities of the physical activity set, a selected physical activity that is associated with a selected environmental metric; and

identifies the physical activity by;

invoking the classifier logic to identify the selected physical activity associated with the selected environmental metric.

12. The system of claim 11, wherein:

the environmental metric is included in an environmental metric set of at least two environmental metrics detected by the environmental sensor; and

the system further comprising at least one training data set that identifies, among the at least two environmental metrics of the environmental metric set, a selected environmental metric that identifies a selected physical activity; and

a classifier logic trainer that trains the classifier logic to identify, for respective training data sets, the selected physical activity identified by the training data set for the selected environmental metric.

13. The system of claim 11, wherein:

the environmental metric is included in an environmental metric set of at least two environmental metrics detected by the environmental sensor;

respective environmental metrics of the environmental metric set identify the physical activity when performed by a particular user while attached to the wearable device; and

the system further comprises a classifier logic trainer that, responsive to determining that, among the at least two environmental metrics of the environmental metric set, a selected environmental metric is detected by the environmental sensor when the physical activity is performed by the particular user, updates the classifier logic to identify the physical activity of the particular user associated with the selected environmental metric.

14. The system of claim 11, wherein:

the classifier logic, in addition to selecting the physical activity, identifies a selection confidence of selecting the physical activity associated with the selected environmental metric; and

the physical activity identifier identifies the performance of the physical activity by invoking the classifier logic to identify the physical activity associated with the at least one environmental metric and having a selection confidence above a selection confidence threshold.

15. The system of claim 14, wherein, responsive to the classifier logic failing to identify a physical activity having a selection confidence above the selection confidence threshold:

the physical activity receiver receives at least one additional environmental metric from the environmental sensor; and

the physical activity identifier invokes the classifier logic to identify the physical activity associated with the at least one environmental metric and the at least one additional environmental metric.

16. The system of claim 11, wherein the physical activity identifier updates the classifier logic to identify the physical activity associated with the selected environmental metric.

17. The system of claim 16, further comprising a physical activity identifying service interface that:

receives the classifier logic from a physical activity identifying service; and

responsive to receiving the user identification of the physical activity, sends the user identification and the at least one environmental metric to the physical activity identifying service.

18. A method of presenting a user interface of an application on a wearable electronic device according to detected physical activities performed by a user wearing the wearable electronic device, comprising:

executing on a processor of the wearable electronic device, instructions that cause the wearable electronic device to;

identify, according to classifier logic, a first detected physical activity that is associated with a first detected environmental metric;

adapting the user interface of the application to have a first set of features based on the first detected physical activity;

identify, according to the classifier logic, a second detected physical activity that is associated with a second detected environmental metric that is different than the first environmental metric; and

adapting the user interface of the application to have a second set of features based on the second detected physical activity.

19. The method of claim 18, further comprising updating the classifier logic according to one of the first physical activity and the first environmental metric or the second physical activity and the second environmental metric.

20. The method of claim 18, wherein adapting the user interface comprises adapting a size of one or more user interface elements provided on the user interface.