Automatic exercise segmentation and recognition

• US 9,174,084 B2
• Filed: 03/05/2013
• Issued: 11/03/2015
• Est. Priority Date: 03/05/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Morris, Daniel, Kelner, Ilya, Shariff, Farah, Tom, Dennis, Saponas, T. Scott, Guillory, Andrew
• Primary Examiner(s): YEN, JASON TAHAI
• Application Number:
  US13/785,184
• Publication Number:
  US 20140257533A1
• Time in Patent Office: 973 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/02438 with portable devices, e.g. worn by the patient
• CPC Class Codes: A61B 5/1112 Global tracking of patients, e.g. by using GPS
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/6824 Arm or wrist
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A63B 2071/0663 worn on the wrist, e.g. wrist bands
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 24/00 Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
• CPC Class Codes: A63B 24/0062 Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A physical activity monitoring device includes a sensor array with one or more sensors configured to measure physical activity attributes of a user. A controller automatically determines time intervals where the user is actively engaged in a physical activity based on the physical activity attributes. The controller also automatically determines a type of physical activity the user in actively engaged in during the determined time intervals based on the physical activity attributes. A reporter outputs information regarding the type of physical activity to the user.

Claims

1. A physical activity monitoring device, comprising:

a sensor array including one or more sensors configured to measure physical activity attributes of a user while the user is wearing the physical activity monitoring device;

a controller operable to;

receive signal information from the sensor array;

divide the signal information into overlapping segments;

identify predetermined signal characteristics for each overlapping segment;

analyze the predetermined signal characteristics for each overlapping segment using a supervised classifier trained to recognize if the user is actively engaged in a physical activity during the overlapping segment;

automatically determine time intervals where the user is actively engaged in the physical activity using the physical activity attributes; and

automatically determine a type of physical activity the user is actively engaged in during the determined time intervals using the physical activity attributes; and

a reporter to output information regarding the type of physical activity.

2. The monitoring device of claim 1, where the one or more sensors include an accelerometer.

3. The monitoring device of claim 1, wherein the signal information includes signals in three dimensions, and wherein the controller is operable to dimensionally reduce signals in two of the three dimensions into a signal in one dimension.

4. The monitoring device of claim 1, wherein the predetermined signal characteristics include one or more of:

a number of autocorrelation peaks, a number of negative autocorrelation peaks, a maximum autocorrelation value, a log of a maximum autocorrelation value, a root-mean-square amplitude, a mean, a standard deviation, a variance, or an integrated root-mean-square amplitude.

5. The monitoring device of claim 1, wherein the supervised classifier includes a support vector machine, and wherein to analyze the predetermined signal characteristics the controller is operable to:

train the support vector machine with data collected from a plurality of users during time intervals where the users were actively engaged in a physical activity and time intervals where the users were not actively engaged in a physical activity;

generate a set of transformation vectors, a weight vector and a threshold representative of the user actively engaged in a physical activity;

multiply the predetermined signal characteristics by the set of transformation vectors and weight vector to obtain a plurality of multiplication products;

compare the multiplication products to the threshold;

classify a value above the threshold as representative of an overlapping segment wherein the user is actively engaged in a physical activity;

classify a value below the threshold as representative of an overlapping segment wherein the user is not actively engaged in a physical activity; and

classify overlapping segments as being representative of time intervals where the user is likely to be actively engaged in a physical activity based on the classified values.

6. The monitoring device of claim 5, further comprising an aggregator configured to determine a time interval defined by a plurality of the classified overlapping segments where the user is likely to be actively engaged in a physical activity.

7. The monitoring device of claim 6, wherein the controller is operable to receive signal information from the sensor array and further operable to receive a set of time intervals from the aggregator, and wherein to determine a type of physical activity the user is actively engaged in during the set of time intervals, the controller is operable to:

divide the signal information into overlapping segments;

identify predetermined signal characteristics for each overlapping segment; and

analyze the predetermined signal characteristics for each overlapping segment using a supervised classifier trained to recognize the type of physical activity the user is actively engaged in during the overlapping segment.

8. The monitoring device of claim 7, where the supervised classifier includes a support vector machine, and wherein to analyze the predetermined signal characteristics, the controller is further operable to:

train the support vector machine with data collected from a plurality of users during time intervals where the users were engaged in a plurality of types of physical activity;

generate a set of transformation vectors and a weight vector representative of a user engaged in a type of physical activity;

multiply the predetermined signal characteristics by the set of transformation vectors and weight vector to obtain a plurality of multiplication products;

compare the multiplication products to data sets representative of each of a plurality of predetermined activities where the data sets have been predetermined through machine learning; and

classify overlapping segments as representative of a type of physical activity.

9. The monitoring device of claim 7, further comprising a voting machine configured to determine the type of physical activity the user is likely to be actively engaged in during a time interval where the aggregator has determined that the user is engaged in a physical activity.

10. The monitoring device of claim 1, wherein the controller is further operable to determine a number of repetitions the user performs of a repetitive physical activity.

11. The monitoring device of claim 10, wherein the controller is operable to determine the number of repetitions through a counting method that includes:

receiving a signal from the sensor array; and

transforming the signal into a dimensionally reduced signal having at least one fewer dimensions than the signal received from the sensor array;

counting a number of peaks of the dimensionally reduced signal; and

outputting the number of peaks.

12. The monitoring device of claim 11, wherein the counting method further includes:

determining a set of candidate peaks;

filtering the set of candidate peaks using local period estimates;

filtering the set of candidate peaks using amplitude statistics; and

counting a number of peaks from the set of candidate peaks.

13. The monitoring device of claim 12, wherein the counting method further includes:

determining a set of candidate valleys;

filtering the set of candidate valleys using local period estimates;

filtering the set of candidate valleys using amplitude statistics;

counting a number of valleys from the set of candidate valleys;

comparing the number of valleys to the number of peaks; and

designating the greater of the number of valleys and the number of peaks as a number of repetitions; and

outputting the number of repetitions.

14. A method of monitoring physical activity, comprising:

measuring, with a sensor array including one or more sensors, physical activity attributes of a user wearing a physical activity monitoring device including the one or more sensors;

automatically determining a set of time intervals where the user is actively engaged in a physical activity based on the physical activity attributes by;

dividing signal information from the sensor array into overlapping segments;

identifying predetermined signal characteristics for each overlapping segment; and

analyzing the predetermined signal characteristics for each overlapping segment using a supervised classifier;

using the supervised classifier to automatically determine a type of physical activity the user is actively engaged in during the determined time intervals based on the physical activity attributes; and

outputting information regarding the type of physical activity.

15. The method of claim 14, further comprising outputting information regarding a form of the user performing the physical activity.

16. The method of claim 14, where the supervised classifier includes a support vector machine, and where analyzing the predetermined signal characteristics further includes:

training the support vector machine with data collected from a plurality of users during time intervals where the users were engaged in a plurality of types of physical activity;

generating a set of transformation vectors and a weight vector representative of a user engaged in a type of physical activity;

multiplying the predetermined signal characteristics by the set of transformation vectors and weight vector to obtain a plurality of multiplication products;

comparing the multiplication products to data sets representative of each of a plurality of predetermined activities where the data sets have been predetermined through machine learning; and

classifying overlapping segments as representative of a type of physical activity.

17. The method of claim 14, further comprising determining the type of physical activity the user is likely to be actively engaged in during a time interval where an aggregator has determined that the user is engaged in a physical activity.

18. A physical activity monitoring device, comprising:

a sensor array including an accelerometer configured to measure physical activity attributes of a user wearing the physical activity monitoring device;

a controller operable to receive acceleration signal information from the sensor array and to automatically determine time intervals where the user is actively engaged in a physical activity by;

dividing the signal information into overlapping segments;

identifying predetermined acceleration characteristics for each overlapping segment; and

analyzing the predetermined acceleration characteristics for each overlapping segment using a supervised classifier trained to recognize if the user is actively engaged in the physical activity during the overlapping segment;

the controller further operable to automatically determine a type of physical activity the user is actively engaged in during the determined time intervals using the physical activity attributes corresponding to the determined time intervals; and

a reporter to output information regarding the type of physical activity.