Method, apparatus and system for food intake and physical activity assessment

• US 9,198,621 B2
• Filed: 06/18/2008
• Issued: 12/01/2015
• Est. Priority Date: 06/18/2007
• Status: Active Grant

• Current Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Original Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Inventors: Fernstrom, John D., Fernstrom, Madelyn H., Sclabassi, Robert J., Sun, Mingui
• Primary Examiner(s): Thomson, William
• Assistant Examiner(s): JIAN, SHIRLEY XUEYING
• Application Number:
  US12/141,741
• Publication Number:
  US 20090012433A1
• Time in Patent Office: 2,722 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2503/06 Children, e.g. for attention deficit diagnosis
• CPC Class Codes: A61B 2505/07 Home care
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/1112 Global tracking of patients, e.g. by using GPS
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/145 Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissuemeasuring of blood pressure or blood flow A61B5/02; non-radiation detecting or locating of foreign bodies in blood A61B5/06
• CPC Class Codes: A61B 5/411 Detecting or monitoring allergy or intolerance reactions to an allergenic agent or substance
• CPC Class Codes: A61B 5/6822 Neck
• CPC Class Codes: A61B 5/7232 involving compression of the physiological signal, e.g. to extend the signal recording period
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G16H 40/67 for remote operation

Abstract

Electronic systems, devices and methods are provided to accurately measure both food intake and physical activity in a subject. A device is provided to be placed on a subject which records video as well as other physiological and/or environmental data. Video data can be analyzed along with other data obtained by the device to determine food consumption and physical activity of the subject, much of which is accomplished by automated computer-implemented processes.

Claims

1. A system for remote monitoring of food intake and physical activity in a human subject, comprising:

a device for remote monitoring of food intake and physical activity in a subject comprising

a housing having a first side and a second side, wherein when the device is worn by the subject, the first side is proximal to the subject and the second side is distal to the subject;

one or more physiological sensors contained within the housing, contained within or connected to the housing, wherein the physiological sensors are selected from the group consisting of a microphone, an oxygen saturation sensor, a heart rate monitor, and an electrode;

a sensor for determining location, the sensor contained within the housing;

a motion sensor contained within the housing;

a video camera contained within the housing;

a data transfer interface contained within the housing and connected to the sensors and the video camera; and

one or more power supplies contained within the housing;

a computer; and

a data communication device for transmitting data obtained from the device to the computer,

wherein the computer is programmed or configured to perform a process for recognizing a swallowing event by a person wearing the device and turning on video recording by the device when data is received by the device that corresponds to a swallowing event, the computer also programmed or configured to perform one or more processes for recognizing a food item, recognizing a physical activity, and determining a dimension of a food item in video data obtained from the video camera and for characterizing or quantifying physical activity of a subject wearing the device in data obtained from one or more of the video camera, the physiological sensor, the sensor for determining location, and the motion sensor.

2. The system of claim 1, in which the device further comprises one or more environmental monitoring sensors or an optical sensor connected to the data transfer interface.

3. The system of claim 2, in which the environmental monitoring sensors include one or more of a light sensor for assessing whether the device is located indoors or outdoors and a thermometer and a humidity sensor for monitoring weather conditions.

4. The system of claim 1, in which the computer is further programmed or configured for distinguishing data associated with food and drink consumption from other data obtained from one or both of the microphone and motion sensor;

and a process for distinguishing data associated with physical activity from other data obtained from the device.

5. The system of claim 1, further comprising a database comprising dietary information for different foods, and a computer process for determining nutritional information from the computer processes for recognizing a food item and for determining a dimension of a food item in video data obtained from the video camera.

6. The system of claim 1, wherein the data transfer interface comprises a device for storing data on storage media located within the housing of the device for storing data from the physiological sensors and video camera.

7. The system of claim 1, further comprising a controller unit for processing data acquired from the one or more physiological sensors and video camera and for managing the data acquisition of the sensors and video camera.

8. The system of claim 1, in which the video camera can be turned on and off by the subject.

9. The system of claim 1, in which the physical activity of the subject is determined by comparing data from the physiological sensors to predetermined activity correlated variables defining specific physical activities.

10. The system of claim 1, comprising:

one or both of a motion sensor and a heart rate sensor within or connected to the housing and connected to the data transfer interface;

sensor for determining location within the housing and connected to the data transfer interface;

one or more environmental monitoring sensors within the housing and connected to the data transfer interface;

an optical sensor within the housing and connected to the data transfer interface;

an oxygen saturation sensor connected to the housing and the data transfer interface;

a controller unit which manages the acquisition of data from the sensors; and

a data storage device for storing data on storage media connected to the sensors and the video camera;

wherein the data transfer interface is connected to the data storage device for transferring data to an external source.

11. The system of claim 1, in which the system comprises a process for recognizing a physical activity and data from the motion sensor corresponding to the physical activity triggers recording from the video camera.

12. The system of claim 1, in which the first side of the housing comprises an acoustic coupling portion, the device comprises a microphone acoustically coupled to the first side of the housing, and data from the microphone corresponding to a swallowing event triggers recording from the video camera.

13. The system of claim 1, further comprising a data encryption process for encrypting data obtained from the device.

14. The system of claim 1, further comprising a process for obscuring facial features of a person in video data obtained from the video camera.

15. The system of claim 1, in which the motion sensor is an accelerometer.

16. The system of claim 1, in which the sensor for determining location is a global positioning sensor.

17. The system of claim 1, in which the computer is programmed or configured to recognize a physical activity by correlating one-dimensional vectors.

18. The system of claim 17, in which the computer is programmed or configured to recognize physical activity by cross-correlating the one-dimensional vectors, using the algorithm: