Method and apparatus for tracking of food intake and other behaviors and providing relevant feedback

• US 10,373,716 B2
• Filed: 01/30/2017
• Issued: 08/06/2019
• Est. Priority Date: 01/28/2016
• Status: Active Grant

• Current Assignee: Medtronic Minimed Incorporated (Medtronic Plc)
• Original Assignee: KLUE Incorporated (Medtronic Plc)
• Inventors: Vleugels, Katelijn, Marianetti, II, Ronald
• Primary Examiner(s): Tomaszewski, Michael
• Application Number:
  US15/419,996
• Publication Number:
  US 20170220772A1
• Time in Patent Office: 918 Days
• Field of Search: None
• CPC Class Codes: A61B 2505/07 Home care
• CPC Class Codes: A61B 2560/0209 adapted for power saving
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1121 Determining geometric values, e.g. centre of rotation or angular range of movement
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/4866 Evaluating metabolism A61B5/083 takes precedence
• CPC Class Codes: A61B 5/4875 Hydration status, fluid retention of the body
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: G16H 70/60 relating to pathologies

Abstract

A sensing device monitors and tracks food intake events and details. A processor, appropriately programmed, controls aspects of the sensing device to capture data, store data, analyze data and provide suitable feedback related to food intake. More generally, the methods might include detecting, identifying, analyzing, quantifying, tracking, processing and/or influencing, related to the intake of food, eating habits, eating patterns, and/or triggers for food intake events, eating habits, or eating patterns. Feedback might be targeted for influencing the intake of food, eating habits, or eating patterns, and/or triggers for those. The sensing device can also be used to track and provide feedback beyond food-related behaviors and more generally track behavior events, detect behavior event triggers and behavior event patterns and provide suitable feedback.

Claims

1. A method of sensing wearer activity, including at least one food intake event, using at least one electronic device worn by a wearer, the method comprising:

determining, using a processor, sensor readings, wherein at least one sensor reading is from an accelerometer of the at least one electronic device that measures movement of an arm of the wearer;

determining a first gesture indicative of a potential event from the sensor readings;

determining, using the processor, a stored state, the stored state being a state from a set of states that includes an in-event state and an out-of-event state;

when in the out-of-event state, determining, using the processor and based on the first gesture, a start of the at least one food intake event;

providing storage for event-specific parameters, initialized for the food intake event;

determining, when in the in-event state, a sequence of gestures related to the food intake event, using the processor, wherein the processor processes gesture indications differently in the in-event state and in the out-of-event state;

changing, using the processor, the electronic device to a higher performance state in response to the stored state being modified from the out-of-event state to the in-event state, wherein the higher performance state comprises one or more of additional power supplied to sensors, additional power being drawn by the sensors, a reduced latency of a communications channel, an increased sensor sampling rate of the sensors, additional computations performed by the processor, activation of a data recorder, and/or sending a signal to an external device indicating one or more of the event-specific parameters; and

deriving the event-specific parameters about the food intake event from the sequence of gestures, the event-specific parameters including a timestamp relative to the start of the food intake event.

2. The method of claim 1, further comprising:

detecting, using the processor and based on at least some of the sensor readings, an end of the food intake event; and

putting the electronic device into a lower performance state.

3. The method of claim 1, wherein determining sensor readings comprises receiving signals from one or more of the accelerometer and additional sensors for detecting the start of the food intake event and wherein the additional sensors include a gyroscope.

4. The method of claim 1, further comprising training a learning engine to predict an occurrence of a food intake event based on tracking of one or more of gender, age, weight, social-economic status, timing information about the food intake event, information about location of food intake event, vital signs information, and hydration level information.

5. The method of claim 1, further comprising:

identifying, using a learning engine, signatures of event time envelopes, thereby delimiting an event time envelope;

identifying, using limits of the event time envelope, gestures within the event time envelope; and

deriving, from the gestures, the food intake event.

19. The method of claim 1, further comprising outputting the timestamp in a first message to a medication management system that dispenses medication to the wearer timed with food eating events or sending a second message to the medication management system at a predetermined time relative to the start of the food intake event.

6. A method of sensing wearer activity, including at least one behavior event, using at least one electronic device worn by a wearer, the method comprising:

determining, using a processor, sensor readings, wherein at least one sensor reading is from an accelerometer of the at least one electronic device that measures movement of an arm of the wearer;

determining a first gesture indicative of a potential behavior event from the sensor readings;

determining a confidence level related to the first gesture, wherein the confidence level relates to a level of confidence that the first gesture was correctly detected;

determining, using the processor, a stored state, the stored state being a state from a set of states that includes an in-event state and an out-of-event state;

when in the out-of-event state, determining, using the processor and based on the first gesture and the confidence level, a start of the at least one behavior event;

providing storage for event-specific parameters, initialized for the behavior event;

when the confidence level is below a threshold, identifying subsequent gestures to determine the start of the behavior event, using the processor, wherein the processor processes gesture indications differently in the in-event state and in the out-of-event state;

determining, when in the in-event state, a sequence of gestures related to the behavior event;

changing, using the processor, the electronic device to a higher performance state in response to the stored state being modified from the out-of-event state to the in-event state, wherein the higher performance state comprises one or more of additional power supplied to sensors, additional power being drawn by the sensors, a reduced latency of a communications channel, an increased sensor sampling rate of the sensors, additional computations performed by the processor, activation of a data recorder, and/or sending a signal to an external device indicating one or more of the event-specific parameters; and

deriving the event-specific parameters about the behavior event from the sequence of gestures, the event-specific parameters including a timestamp for the start of the behavior event.

7. The method of claim 6, further comprising:

detecting, using the processor and based on at least some of the sensor readings, an end of the behavior event; and

putting the electronic device into a lower performance state.

8. The method of claim 6, wherein determining sensor readings comprises receiving signals from one or more of the accelerometer and additional sensors for detecting the start of the behavior event and wherein the additional sensors include a gyroscope.

9. The method of claim 6, wherein the behavior event is an eating event or a drinking event, the method further comprising estimating one or more of a duration of the behavior event, a bite count, a sip count, an eating pace, or a drinking pace.

10. The method of claim 6, further comprising recording incidences of triggers that autonomously predict a probable start of a food intake event.

11. The method of claim 10, further comprising:

training a learning engine to predict an occurrence of the behavior event based on tracking of one or more of gender, age, weight, social-economic status, timing information about the at least one behavior event, information about location of behavior event, and vital signs information;

identifying, using the learning engine, signatures of event time envelopes, thereby delimiting an event time envelope;

identifying, using limits of the event time envelope, gestures within the event time envelope; and

deriving, from the gestures, the at least one behavior event.

12. The method of claim 6, further comprising:

upon detection of a possible gesture, if the confidence level is below the threshold, waiting for detection of another gesture within a predefined time window following the detection of the possible gesture before determining that the start of the behavior event had occurred.

20. The method of claim 6, further comprising outputting the timestamp in a first message to the wearer timed with behavior events or sending a second message to the wearer at a predetermined time relative to the start of the behavior event.

13. An electronic system for sensing user activity, including at least one behavior event of a user, comprising:

a wearable device having an accelerometer that measures movement of an arm of the user when the wearable device is being worn by the user;

one or more additional sensors that, when the wearable device is being worn by the user, sense parameters about movement of the user;

a processor capable of reading sensor readings of the parameters, wherein at least one sensor reading is from the accelerometer;

memory storage for a stored state, the stored state being a state from a set of states that includes an in-event state and an out-of-event state;

program code executable by the processor for determining, when in the out-of-event state, based on at least some of the sensor readings, a first gesture indicative of a potential start of the at least one behavior event and a confidence level for the first gesture;

program code executable by the processor for determining, when in the in-event state, a sequence of gestures, using sensor readings, wherein the processor processes gesture indications differently in the in-event state and in the out-of-event state;

program code executable by the processor for changing to a higher performance state in response to the stored state being modified from the out-of-event state to the in-event state, wherein the higher performance state comprises one or more of additional power supplied to sensors, additional power being drawn by the sensors, a reduced latency of a communications channel, an increased sensor sampling rate of the sensors, additional computations performed by the processor, activation of a data recorder, and/or sending a signal to an external device indicating one or more event-specific parameters; and

program code for deriving the event-specific parameters about the behavior event from the sequence of gestures, the event-specific parameters including a timestamp for the potential start of the behavior event.

14. The electronic system of claim 13, further comprising:

program code for detecting a specific gesture from the parameters based in part on the stored state;

a gyroscope sensor; and

program code for determining, from the gyroscope sensor the accelerometer, or both, a possible gesture.

15. The electronic system of claim 13, further comprising:

program code for determining external metadata about a history of the user; and

program code for using the external metadata to improve accuracy of the confidence level.

16. The electronic system of claim 13, further comprising:

memory storage for a learned dataset, wherein the learned dataset comprises data usable to detect a type of gesture based on training examples;

a classifier trained on the learned dataset comprising data records related to prior gestures of the user or other users, wherein the program code for determining possible gestures of the user takes into account outputs of the classifier, whereby the classifier can be used in a food intake event detection system to detect a start of a food intake event, wherein the learned dataset comprises data related to one or of sleep pattern, stress level, daily interactions, and recent activities.

17. The electronic system of claim 13, further comprising:

program code for determining possible gestures of the user;

program code for assigning confidence levels to the possible gestures;

program code for determining the start of a food intake event based on a possible gesture being determined as an eating or drinking gesture above a threshold confidence level;

program code for providing additional power to additional sensors; and

program code for using inputs of the additional sensors to improve accuracy of the confidence level.

18. The electronic system of claim 13, further comprising:

program code for distinguishing eating gestures from drinking gestures based on accelerometer sensor data and gyroscope sensor data, wherein sensing takes into account changes in an angle of roll of a body part of the user and/or variance of accelerometer or gyroscope readings across one or more axes for a duration of time;

program code for distinguishing, based on detected rotation of a wrist of the user, between an eating gesture and a drinking gesture; and

program code for determining, from sensor data, bite counts, sip counts, cadence, and duration, to determine bite sizes and sip sizes, wherein bite sizes and sip sizes are determined based on a duration of a hand of the user near the mouth of the user or determined based on an amount of rotation of the wrist of the user and wherein the sip sizes are used in a hydration tracking process.

21. A method of sensing wearer activity, including a smoking event, using at least one electronic device worn by a wearer, the method comprising:

determining, using a processor, sensor readings from a set of sensors, wherein at least one sensor reading is from an accelerometer of the at least one electronic device that measures movement of an arm of the wearer;

determining a first gesture indicative of a potential event from the sensor readings;

determining, using the processor, a stored state, the stored state being a state from a set of states that includes an in-event state and an out-of-event state;

when in the out-of-event state, determining, using the processor and based on the first gesture, a start of the smoking event;

providing storage for event-specific parameters, initialized for the smoking event;

determining, when in the in-event state, a sequence of gestures related to the smoking event, using the processor, wherein the processor processes gesture indications differently in the in-event state and in the out-of-event state;

derive the event-specific parameters about the smoking event from the sequence of gestures, the event-specific parameters including a timestamp relative to the start of the smoking event;

change, using the processor, the electronic device to a higher performance state in response to the stored state being modified from the out-of-event state to the in-event state, wherein the higher performance state comprises one or more of additional power supplied to sensors of the set of sensors, additional power being drawn by the sensors of the set of sensors, a reduced latency of a communications channel, an increased sensor sampling rate of the sensors of the set of sensors, additional computations performed by the processor, activation of a data recorder, and/or sending a signal to an external device indicating one or more of the event-specific parameters; and

output the timestamp in a message to the wearer or send the message to the wearer at a predetermined time relative to the start of the smoking event.