WEARABLE SYSTEM FOR PREDICTING ABOUT-TO-EAT MOMENTS

• US 20170172493A1
• Filed: 12/17/2015
• Published: 06/22/2017
• Est. Priority Date: 12/17/2015
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Czerwinski, Mary, Rahman, Tauhidur, Gilad-Bachrach, Ran, Rowan, Kael Robert, Johns, Paul R., Roseway, Asta
• Application Number:
  US14/973,645
• Publication Number:
  US 20170172493A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/486 Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/6801 specially adapted to be attached to or worn on the body surface
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7275 Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
• CPC Class Codes: A61B 5/7405 using sound
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 5/7455 characterised by tactile indication, e.g. vibration or electrical stimulation
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: A63B 2220/12 Absolute positions, e.g. by using GPS
• CPC Class Codes: A63B 2220/17 Counting, e.g. counting periodical movements, revolutions or cycles, or including further data processing to determine distances or speed
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/34 Angular speed
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2230/06 heartbeat rate only
• CPC Class Codes: A63B 2230/50 temperature
• CPC Class Codes: A63B 2230/65 skin conductivity
• CPC Class Codes: A63B 2230/75 calorie expenditure
• CPC Class Codes: A63B 24/0062 Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: G09B 19/0092 : Nutrition
• CPC Class Codes: G16H 20/60 : relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A system is provided that predicts eating events for a user. The system includes a set of sensors each of which is configured to continuously measure a different physiological variable associated with the user and output a time-stamped data stream that includes the current value of this variable. A set of features is periodically extracted from the data stream output from each of the sensors, where these features have been determined to be specifically indicative of an about-to-eat moment. This set of features is then input into an about-to-eat moment classifier that has been trained to predict when the user is in an about-to-eat moment based on this set of features. Whenever an output of the classifier indicates that the user is currently in an about-to-eat moment, the user is notified with a just-in-time eating intervention.

Claims

1. A system for predicting eating events for a user, comprising:

a set of mobile sensors, each of the mobile sensors being configured to continuously measure a different physiological variable associated with the user and output a time-stamped data stream comprising the current value of said variable; and

an eating event forecaster comprising one or more computing devices, said computing devices being in communication with each other via a computer network whenever there is a plurality of computing devices, and a computer program having a plurality of sub-programs executable by the one or more computing devices, the one or more computing devices being directed by the sub-programs of the computer program to,

for each of the mobile sensors,

receive the data stream output from the mobile sensor, and

periodically extract a set of features from said received data stream, said features, which are among many features that can be extracted from said received data stream, having been determined to be specifically indicative of an about-to-eat moment,

input the set of features that is periodically extracted from the data stream received from each of the mobile sensors into an about-to-eat moment classifier that has been trained to predict when the user is in an about-to-eat moment based on said set of features, and

whenever an output of the classifier indicates that the user is currently in an about-to-eat moment, notify the user with a just-in-time eating intervention.

2. The system of claim 1, wherein the mobile sensors comprise one or more of:

a wearable computing device that is physically attached to the body of the user and provides health and fitness tracking functionality for the user;

or

a mobile computing device that is carried by the user.

3. The system of claim 1, wherein the mobile sensors comprise one or more of:

a heart rate sensor that is physically attached to the body of the user;

or

a skin temperature sensor that is physically attached to the body of the user;

or

an accelerometer that is physically attached to or carried by the user;

or

a gyroscope that is physically attached to or carried by the user;

or

a global positioning system sensor that is physically attached to or carried by the user;

or

an electrodermal activity sensor that is physically attached to the body of the user;

or

a body conduction microphone that is physically attached to the body of the user.

4. The system of claim 1, wherein the classifier comprises one of:

a linear type classifier;

or

a reduced error pruning type classifier;

or

a support vector machine type classifier;

or

a TreeBagger type classifier.

5. The system of claim 1, wherein,

one of the computing devices comprises a mobile computing device that is carried by the user, and

said user notification comprises one or more of:

a message that is displayed on a display screen of the mobile computing device;

or

an audible alert that is output from the mobile computing device;

or

a haptic alert that is output from the mobile computing device.

6. The system of claim 1, wherein said received data stream comprises one of:

the current heart rate of the user;

or

the current skin temperature of the user;

or

the current three-dimensional linear velocity of the user;

or

the current three-dimensional angular velocity of the user;

or

the current longitude of the user;

or

the current latitude of the user;

or

the current electrodermal activity of the user;

or

current non-speech body sounds that are conducted through the body surface of the user, said sounds comprising the chewing and swallowing sounds of the user;

or

a current cumulative value for the step count of the user;

or

a current cumulative value for the calorie expenditure of the user;

or

the current speed of movement of an arm of the user.

7. The system of claim 1, wherein the sub-program for periodically extracting a set of features from said received data stream comprises sub-programs for:

preprocessing said received data stream; and

periodically extracting the set of features from the preprocessed received data stream, said periodic extraction comprising sub-programs for,

segmenting the preprocessed received data stream into windows each of which comprises a prescribed uniform window length and a prescribed uniform window shift, and

applying a set of statistical functions to each of said windows, each of the statistical functions extracting a different feature from each of said windows.

8. The system of claim 7, wherein the sub-program for preprocessing said received data stream comprises sub-programs for:

whenever said received data stream comprises the current three-dimensional linear velocity of the user, normalizing said received data stream;

whenever said received data stream comprises the current three-dimensional angular velocity of the user, normalizing said received data stream;

whenever said received data stream comprises a current cumulative value for the step count of the user, interpolating said received data stream, and using differentiation on the interpolated received data stream to estimate an instantaneous value for the step count of the user at each point in time;

whenever said received data stream comprises a current cumulative value for the calorie expenditure of the user, interpolating said received data stream, and using differentiation on the interpolated received data stream to estimate an instantaneous value for the calorie expenditure of the user at each point in time;

whenever said received data stream comprises the current electrodermal activity of the user, computing the mean of said received data stream, subtracting said mean from said received data stream, and decomposing the resulting data stream into a slow-varying tonic component and a fast-varying phasic component; and

whenever said received data stream comprises current non-speech body sounds that are conducted through the body surface of the user, detecting each of the eating events in said received data stream.

9. The system of claim 8, wherein the sub-program for detecting each of the eating events in said received data stream comprises a sub-program for using a BodyBeat mastication and swallowing sound detection method to detect characteristic eating sounds in said received data stream.

10. The system of claim 7, wherein the set of features that is periodically extracted from the preprocessed received data stream comprises two or more of:

the minimum data value within each of said windows;

or

the maximum data value within each of said windows;

or

the mean data value within each of said windows;

or

the root mean square data value within each of said windows;

or

the first quartile of the data within each of said windows;

or

the second quartile of the data within each of said windows;

or

the third quartile of the data within each of said windows;

or

the standard deviation of the data within each of said windows;

or

the interquartile range of the data within each of said windows;

or

the total number of data peaks within each of said windows;

or

the mean distance between successive data peaks within each of said windows;

or

the mean amplitude of the data peaks within each of said windows;

or

the mean crossing rate of the data within each of said windows;

or

the linear regression slope of the data within each of said windows;

or

the time that has elapsed since the beginning of the day for the user;

or

the time that has elapsed since the last eating event for the user;

or

the number of previous eating events for the user since the beginning of the day for the user.

11. A system for predicting eating events for a user, comprising:

a set of mobile sensors, each of the mobile sensors being configured to continuously measure a different physiological variable associated with the user and output a time-stamped data stream comprising the current value of said variable; and

an eating event forecaster comprising one or more computing devices, said computing devices being in communication with each other via a computer network whenever there is a plurality of computing devices, and a computer program having a plurality of sub-programs executable by the one or more computing devices, the one or more computing devices being directed by the sub-programs of the computer program to,

for each of the mobile sensors,

receive the data stream output from the mobile sensor, and

periodically extract a set of features from said received data stream, said features, which are among many features that can be extracted from said received data stream, having been determined to be specifically indicative of an about-to-eat moment,

input the set of features that is periodically extracted from the data stream received from each of the mobile sensors into a regression-based time-to-next-eating-event predictor that has been trained to predict the time remaining until the onset of the next eating event for the user based on said set of features, and

whenever an output of the predictor indicates that the current time remaining until the onset of the next eating event for the user is less than a prescribed threshold, notify the user with a just-in-time eating intervention.

12. The system of claim 11, wherein the predictor comprises one of:

a linear type predictor;

or

a reduced error pruning type predictor;

or

a sequential minimal optimization type predictor;

or

a TreeBagger type predictor.

13. The system of claim 11, wherein,

one of the computing devices comprises a mobile computing device that is carried by the user, and

said user notification comprises one or more of:

a message that is displayed on a display screen of the mobile computing device;

or

an audible alert that is output from the mobile computing device;

or

a haptic alert that is output from the mobile computing device.

14. The system of claim 11, wherein said received data stream comprises one of:

the current heart rate of the user;

or

the current skin temperature of the user;

or

the current three-dimensional linear velocity of the user;

or

the current three-dimensional angular velocity of the user;

or

the current longitude of the user;

or

the current latitude of the user;

or

the current electrodermal activity of the user;

or

current non-speech body sounds that are conducted through the body surface of the user, said sounds comprising the chewing and swallowing sounds of the user;

or

a current cumulative value for the step count of the user;

or

a current cumulative value for the calorie expenditure of the user;

or

the current speed of movement of an arm of the user.

15. The system of claim 11, wherein the sub-program for periodically extracting a set of features from said received data stream comprises sub-programs for:

preprocessing said received data stream; and

periodically extracting the set of features from the preprocessed received data stream, said periodic extraction comprising sub-programs for,

segmenting the preprocessed received data stream into windows each of which comprises a prescribed uniform window length and a prescribed uniform window shift, and

applying a set of statistical functions to each of said windows, each of the statistical functions extracting a different feature from each of said windows.

16. The system of claim 15, wherein the set of features that is periodically extracted from the preprocessed received data stream comprises two or more of:

the minimum data value within each of said windows;

or

the maximum data value within each of said windows;

or

the mean data value within each of said windows;

or

the root mean square data value within each of said windows;

or

the first quartile of the data within each of said windows;

or

the second quartile of the data within each of said windows;

or

the third quartile of the data within each of said windows;

or

the standard deviation of the data within each of said windows;

or

the interquartile range of the data within each of said windows;

or

the total number of data peaks within each of said windows;

or

the mean distance between successive data peaks within each of said windows;

or

the mean amplitude of the data peaks within each of said windows;

or

the mean crossing rate of the data within each of said windows;

or

the linear regression slope of the data within each of said windows;

or

the time that has elapsed since the beginning of the day for the user;

or

the time that has elapsed since the last eating event for the user;

or

the number of previous eating events for the user since the beginning of the day for the user.

17. A system for training a machine-learned eating event predictor, comprising:

a set of mobile sensors, each of the mobile sensors being configured to continuously measure a different physiological variable associated with each of one or more users and output a time-stamped data stream comprising the current value of said variable; and

an eating event prediction trainer comprising one or more computing devices, said computing devices being in communication with each other via a computer network whenever there is a plurality of computing devices, and a computer program having a plurality of sub-programs executable by the one or more computing devices, the one or more computing devices being directed by the sub-programs of the computer program to,

for each of the mobile sensors,

receive the data stream output from the mobile sensor, and

periodically extract a set of features from said received data stream, said features, which are among many features that can be extracted from said received data stream, having been determined to be specifically indicative of an about-to-eat moment,

use the set of features that is periodically extracted from the data stream received from each of the mobile sensors to train the predictor to predict when an eating event for a user is about to occur, and

output the trained predictor.

18. The system of claim 17, wherein the predictor comprises an about-to-eat moment classifier that is trained to predict when a user is in an about-to-eat moment.

19. The system of claim 17, wherein,

the predictor comprises a regression-based time-to-next-eating-event predictor,

the sub-program for periodically extracting a set of features from said received data stream comprises a sub-program for mapping each of the features in the set of features that is periodically extracted from said received data stream to the current time remaining until the next eating event, said current time remaining being determined by analyzing the data stream received from each of the mobile sensors, and

the sub-program for using the set of features that is periodically extracted from the data stream received from each of the mobile sensors to train the predictor to predict when an eating event for a user is about to occur comprises a sub-program for using the set of features that is periodically extracted from the data stream received from each of the mobile sensors in combination with said mapping of each of the features in said set of features to train the time-to-next-eating-event predictor to predict the time remaining until the onset of the next eating event for the user.

20. The system of claim 17, wherein the sub-program for using the set of features that is periodically extracted from the data stream received from each of the mobile sensors to train the predictor to predict when an eating event for a user is about to occur comprises sub-programs for:

inputting the set of features that is periodically extracted from the data stream received from each of the mobile sensors into an overall set of features;

using a combination of a correlation-based feature selection method and a best-first decision tree machine learning method to select a subset of the features in the overall set of features; and

using the selected subset of the features to train the predictor to predict when an eating event for a user is about to occur.