Upsampling sensors to auto-detect a fitness activity

• US 10,416,740 B2
• Filed: 08/26/2015
• Issued: 09/17/2019
• Est. Priority Date: 08/26/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Singh, Satinder Pal, Estephanio de Moura, Gustavo Mendes, Xiao, Ke
• Primary Examiner(s): Schechter, Andrew
• Assistant Examiner(s): Crohn, Mark I
• Application Number:
  US14/836,628
• Publication Number:
  US 20170056722A1
• Time in Patent Office: 1,483 Days
• Field of Search: None
• CPC Class Codes: A61B 2560/0242 adapted to measure environmental factors, e.g. temperature, pollution
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/02438 with portable devices, e.g. worn by the patient
• CPC Class Codes: A61B 5/1112 Global tracking of patients, e.g. by using GPS
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/14532 for measuring glucose, e.g. by tissue impedance measurement
• CPC Class Codes: A61B 5/681 Wristwatch-type devices
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/26 Power supply means, e.g. regulation thereof for memories G11C
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 40/67 for remote operation

Abstract

In some examples, a method includes detecting, based at least in part on sampling a first set (122) of a plurality of sensors of a computing device (110) at a first rate, an indication that the user has initiated a fitness activity (112B), wherein the computing device (110) stores pre-defined identifiers of non-fitness and fitness activities in a set of pre-defined indications of activities; responsive to detecting the indication that the user has initiated the fitness activity, sampling, at a second rate that is greater than the first rate, a second set (120) of the plurality of sensors to determine a probability that the user is engaged in the fitness activity (112B); and responsive to determining that the probability satisfies a threshold, collecting, sensor data for the fitness activity using a particular set of the plurality of sensors that corresponds to a pre-defined identifier for the fitness activity (112B).

Claims

1. A method for conserving power while reducing latency in collecting sufficient information for identifying particular fitness activities being performed by users of computing devices, the method comprising:

determining, by a computing device while operating in a normal operational state of the computing device and based at least in part on applying a first Hidden Markov Model transformation to a sampling of a first set of a plurality of sensors of the computing device taken at a first rate that is a slowest rate from among three sampling rates so as to conserve power, whether a user of the computing device has transitioned from performing a non-fitness activity to initiating any fitness activity, wherein;

the computing device stores pre-defined identifiers of non-fitness and fitness activities in a set of pre-defined indications of activities;

determining whether the user of the computing device has transitioned from performing the non-fitness activity to initiating any fitness activity comprises identifying initial data from the first Hidden Markov Model transformation that precedes subsequent data from the first Hidden Markov Model transformation;

the initial data corresponds to one or more of the pre-defined identifiers of non-fitness activities in the set of predefined indications of activities; and

the subsequent data corresponds to one or more of the pre-defined identifiers of fitness activities in the set of predefined indications of activities;

responsive to determining that the user has transitioned from performing the non-fitness activity to initiating any fitness activity and until a probability that the user is engaged in a particular fitness activity satisfies a threshold, sampling, by the computing device while operating in an upsampling operational state of the computing device that uses more power than the normal operational state of the computing device, at a second rate that is a fastest rate from among the three sampling rates so as to more quickly obtain sufficient sensor data for identifying the particular fitness activities, a second set of the plurality of sensors to collect the sufficient sensor data to determine the probability that the user is engaged in a particular fitness activity, wherein the second set of the plurality of sensors includes a greater quantity of sensors than the first set of the plurality of sensors, wherein the probability that the user is engaged in the particular fitness activity is determined in part by applying a second Hidden Markov Model transformation to the sufficient sensor data;

responsive to determining that the probability satisfies the threshold, collecting, by the computing device in an active operational state of the computing device that uses less power than the upsampling operational state and more power than the normal operating state, at a third rate that is greater than the first rate and less than the second rate so as to efficiently collect fitness information, additional sensor data as fitness information associated with the particular fitness activity, the additional sensor data being collected from a particular set of the plurality of sensors that corresponds to a pre-defined identifier for the particular fitness activity in the set of pre-defined indications of activities; and

outputting, via a user interface of the computing device, the fitness information associated with the particular activity that was collected during the active operational state.

2. The method of claim 1, wherein the method further comprises:

determining, by the computing device, the pre-defined identifier for the particular fitness activity; and

determining, by the computing device, the threshold based at least in part on the pre-defined identifier for the particular fitness activity.

3. The method of claim 1, further comprising:

determining, by the computing device while operating in the normal operational state, the pre-defined identifier for the particular fitness activity; and

determining, by the computing device and based at least in part on the pre-defined identifier for the particular fitness activity, at least one of the second rate for the upsampling operational state or the second set of the plurality of sensors for the upsampling operational state.

4. The method of claim 1, further comprising:

determining, by the computing device while operating in the normal operational state, the pre-defined identifier for the particular fitness activity; and

determining, by the computing device and based at least in part on the pre-defined identifier for the particular fitness activity, at least one of the third rate of sampling the particular set of the plurality of sensors for the active operational state or the particular set of the plurality of sensors for the active operational state.

5. The method of claim 1, further comprising:

determining, by the computing device while operating in the active operational state, whether values, generated by at least one sensor of the particular set of the plurality of sensors that corresponds to the pre-defined identifier, have changed for a defined period of time; and

decreasing, by the computing device and in response to determining that the values have not changed for the defined period of time, the third rate of sampling the at least one sensor.

6. The method of claim 1, wherein the particular fitness activity is a first fitness activity, the method further comprising:

generating, by the computing device while operating in the upsampling operational state, activity recognition results (ARRs) based at least in part on raw sensor component data from the second set of the plurality of sensors, wherein at least one of the ARRs indicates the probability for the first fitness activity;

detecting, by the computing device, an anomaly in the raw sensor component data for a second fitness activity; and

selecting, by the computing device, based at least in part on the at least one of the ARRs and the anomaly in the raw sensor component data, the first fitness activity, wherein a probability of the second fitness activity is greater than the probability for the first fitness activity.

7. The method of claim 1, further comprising:

prior to determining that the user has transitioned from performing the non-fitness activity to initiating any fitness activity, determining, by the computing device, a first pre-defined identifier of an activity engaged in by the user while the computing device is operating in the normal operational state;

further responsive determining that the user has transitioned from performing the non-fitness activity to initiating any fitness activity, determining, by the computing device while operating in the upsampling operational state, that an unknown activity is associated with a highest probability in a set of probabilities associated with the set pre-defined indications of activities;

blacklisting, by the computing device and for a pre-defined period of time, the first pre-defined identifier of the activity; and

responsive to detecting a second pre-defined identifier of an activity engaged in by the user while the computing device is operating in the normal operational state, refraining, while the pre-defined identifier of the activity is blacklisted, from operating in the upsampling operational state, wherein the first and second pre-defined identifiers are the same.

8. The method of claim 7, further comprising, increasing the pre-defined period of time for each consecutive determination by the computing device that the unknown activity is associated with the highest probability in the set of probabilities associated with the set pre-defined indications of activities.

9. The method of claim 1, further comprising:

storing, by the computing device, a battery usage statistic that indicates an amount of power consumed, during the upsampling operational state, by at least one sensor of the plurality of sensors; and

responsive to determining that the at least one battery usage statistic satisfies a threshold, refraining, by the computing device, from sampling the at least one sensor at the second rate in response to determining that the user has transitioned from performing the non-fitness activity to initiating any fitness activity.

10. The method of claim 1, where the threshold is a first threshold and the probability is a first probability, the method further comprising:

responsive to detecting, while operating in the normal operational state, that a second probability of the particular fitness activity is greater than a second threshold, changing operation of the computing device directly to the active operational state from the normal operational state without transitioning to the upsampling operational state, wherein the second threshold is greater than the first threshold.

11. The method of claim 1, further comprising:

responsive to detecting, while in the active operational state, that the user has changed from the particular fitness activity to a different activity, changing operation of the computing device from the active operational state to the upsampling operational state.

12. The method of claim 1, further comprising:

responsive to determining that the user has transitioned from performing the non-fitness activity to initiating any fitness activity, sending, by the computing device and to a wearable computing device, one or more instructions that cause the wearable computing device to sample a third set of sensors of the wearable computing device at the second rate; and

receiving, by the computing device and from the wearable computing device, information from the wearable computing device that is based at least in part on raw sensor component data generated by the wearable computing device at the second rate.

13. The method of claim 12, further comprising:

responsive to determining that the wearable computing device includes at least one sensor of the same type as at least one sensor of the second set of the plurality of sensors, sending, by the computing device and to the wearable computing device, one or more instructions that cause the wearable computing device to refrain from sampling the at least one sensor of the wearable computing device at the second rate.

14. The method of claim 1, wherein determining whether the user of the computing device has transitioned from performing the non-fitness activity to initiating any fitness activity is based at least in part on receiving information from a wearable computing device, wherein the information from the wearable computing device is based at least in part on raw sensor component data generated by the wearable computing device.

15. The method of claim 1, wherein the non-fitness activity comprises at least one of sitting or standing.

16. The method of claim 1, wherein the non-fitness activity comprises walking.

17. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause at least one processor to:

determine, while a computing device is operating in a normal operational state of the computing device and based at least in part on applying a first Hidden Markov Model transformation to a sampling of a first set of a plurality of sensors of the computing device taken at a first rate that is a slowest rate from among three sampling rates so as to conserve power, whether a user of the computing device has transitioned from performing a non-fitness activity to initiating any fitness activity, wherein;

the computing device stores pre-defined identifiers of non-fitness and fitness activities in a set of pre-defined indications of activities;

the at least one processor determines whether the user of the computing device has transitioned from performing the non-fitness activity to initiating any fitness activity comprises identifying initial data from the first Hidden Markov Model transformation that precedes subsequent data from the first Hidden Markov Model transformation;

the initial data corresponds to one or more of the pre-defined identifiers of non-fitness activities in the set of predefined indications of activities; and

the subsequent data corresponds to one or more of the pre-defined identifiers of fitness activities in the set of predefined indications of activities;

responsive to determining that the user has transitioned from performing the non-fitness activity to initiating any fitness activity and until a probability that the user is engaged in a particular fitness activity satisfies a threshold, sample, while the computing device is operating in an upsampling operational state of the computing device that uses more power than the normal operational state of the computing device, at a second rate that is a fastest rate from among the three sampling rates so as to more quickly obtain sufficient sensor data for identifying particular fitness activities, a second set of the plurality of sensors collect the sufficient sensor data to determine the probability that the user is engaged in a particular fitness activity, wherein the second set of the plurality of sensors includes a greater quantity of sensors than the first set of the plurality of sensors, wherein the probability that the user is engaged in the particular fitness activity is determined in part by applying a second Hidden Markov Model transformation to the sufficient sensor data;

responsive to determining that the probability satisfies the threshold, collect, while the computing device is operating in an active operational state of the computing device that uses less power than the upsampling operational state and more power than the normal operating state, at a third rate that is greater than the first rate and less than the second rate so as to efficiently collect fitness information, additional sensor data as fitness information associated with the particular fitness activity, the additional sensor data being collected from a particular set of the plurality of sensors that corresponds to a pre-defined identifier for the fitness activity in the set of pre-defined indications of activities; and

output, via a user interface of the computing device, the fitness information associated with the particular activity that was collected during the active operational state.

18. The non-transitory computer-readable storage medium of claim 17 encoded with instructions that, when executed, cause the at least one processor to:

determine, while the computing device is operating in the normal operational state, the pre-defined identifier for the particular fitness activity; and

determine, based at least in part on the pre-defined identifier for the particular fitness activity, at least one of the second rate for the upsampling operational state or the second set of the plurality of sensors for the upsampling operational state.

19. A computing device comprising:

one or more computer processors;

a plurality of sensors operably coupled to the one or more computer processors; and

a memory comprising instructions that when executed by the one or more computer processors cause the one or more computer processors to;

determine, while the computing device is operating in a normal operational state of the computing device and based at least in part on applying a first Hidden Markov Model transformation to a sampling of a first set of a plurality of sensors of the computing device taken at a first rate that is a slowest rate from among three sampling rates so as to conserve power, whether a user of the computing device has transitioned from performing a non-fitness activity to initiating any fitness activity, wherein;

the computing device stores pre-defined identifiers of non-fitness and fitness activities in a set of pre-defined indications of activities;

the one or more processors determine whether the user of the computing device has transitioned from performing the non-fitness activity to initiating any fitness activity comprises identifying initial data from the first Hidden Markov Model transformation that precedes subsequent data from the first Hidden Markov Model transformation;

the initial data corresponds to one or more of the pre-defined identifiers of non-fitness activities in the set of predefined indications of activities; and

the subsequent data corresponds to one or more of the pre-defined identifiers of fitness activities in the set of predefined indications of activities;

responsive to determining that the user has transitioned from performing the non-fitness activity to initiating any fitness activity and until a probability that the user is engaged in a particular fitness activity satisfies a threshold, sample, while the computing device is operating in an upsampling operational state of the computing device that uses more power than the normal operational state of the computing device, at a second rate that is a fastest rate from among the three sampling rates so as to more quickly obtain sufficient sensor data for identifying particular fitness activities, a second set of the plurality of sensors collect the sufficient sensor data to determine the probability that the user is engaged in a particular fitness activity, wherein the second set of the plurality of sensors includes a greater quantity of sensors than the first set of the plurality of sensors, wherein the probability that the user is engaged in the particular fitness activity is determined in part by applying a second Hidden Markov Model transformation to the sufficient sensor data;

responsive to determining that the probability satisfies the threshold, collect, while the computing device is operating in an active operational state of the computing device that uses less power than the upsampling operational state and more power than the normal operating state, at a third rate that is greater than the first rate and less than the second rate so as to efficiently collect fitness information, additional sensor data as fitness information associated with the particular fitness activity, the additional sensor data being collected from a particular set of the plurality of sensors that corresponds to a pre-defined identifier for the fitness activity in the set of pre-defined indications of activities; and

output, via a user interface of the computing device, the fitness information associated with the particular activity that was collected during the active operational state.

20. The computing device of claim 19, wherein the memory comprises instructions that when executed by the one or more computer processors cause the one or more computer processors to:

determine, while the computing device is operating in the normal operational state, the pre-defined identifier for the particular fitness activity; and

determine, based at least in part on the pre-defined identifier for the particular fitness activity, at least one of the second rate for the upsampling operational state or the second set of the plurality of sensors for the upsampling operational state.