Method, Apparatus and System for Food Intake and Physical Activity Assessment

• US 20130267794A1
• Filed: 11/14/2012
• Published: 10/10/2013
• Est. Priority Date: 11/14/2011
• Status: Active Grant

• Current Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Original Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Inventors: Sclabassi, Robert, Sun, Mingui, Fernstrom, John, Fernstrom, Madelyn, Jia, Wenyan
• Granted Patent: US 10,006,896 B2
• Time in Patent Office: Days
• US Class Current: 600/301
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0024 for multiple sensor units attached to the patient, e.g. using a body or personal area network
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/1112 Global tracking of patients, e.g. by using GPS
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/24 Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
• CPC Class Codes: A61B 5/6802 Sensor mounted on worn items
• CPC Class Codes: A61B 5/6805 Vests
• CPC Class Codes: A61B 5/6816 Ear lobe
• CPC Class Codes: A61B 7/00 Instruments for auscultation
• CPC Class Codes: G01N 21/00 Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light G01N3/00 - G01N19/00 take precedence
• CPC Class Codes: G01N 33/02 Food
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Electronic systems, devices and methods are provided to accurately record and analyze food intake and physical activity in a subject. A device is provided to be placed on a subject which records video using at least two video cameras positioned to record stereoscopic image pair, as well as other physiological and/or environmental data including, for example, oxygen saturation, heart rate, and environmental factors such as physical location, temperature, and humidity. Video data is analyzed along with other data obtained by the device to determine food consumption and/or physical activity of the subject, much of which is accomplished by automated computer-implemented processes.

Claims

1. A system for the remote monitoring of food intake and physical activity in a subject, comprising:

a) a device configured to be worn on a subject for the remote monitoring of food intake and physical activity in a subject, comprising;

a housing;

a bus inside the housing;

one or more sensors connected to the bus;

two or more imaging sensors connected to the bus and positioned in the housing to record a stereo image;

an eating detector in the housing and connected to the bus, eating detector comprising a light beam source configured in the device to pass a light beam in front of a subject'"'"'s mouth when the device is worn by a subject and a light sensor able to measure changes of light intensity in a path of a light beam from the light beam source;

a microprocessor connected to the bus;

a data storage device connected to the bus and comprising a data storage medium;

a program for managing one or more operations of the device including imaging, data acquisition and storage; and

a power supply connected to the bus;

b) a computer; and

c) a data communication interface for transferring data between the device and the computer,

wherein one or both of the device and the computer comprises one or more programs for determining a dimension of a food item in image data obtained from the imaging sensors and determining a physical activity of a subject wearing the device.

2. The system of claim 1, in which the program for determining a dimension of a food item produces a representation of a stereo image of a food item in a three dimensional perspective, scales and orients a computer-generated template shape to fit the food item depicted in the stereo image representation and determines the volume of the computer-generated template thereby estimating the volume of the food item.

3. The system of claim 1, in which the program for determining the physical activity of a subject is automated and comprises comparing data from the one or more physiological sensors and video cameras to known values for various physical activities;

estimating a type of physical activity performed by the subject based on the comparison between the measured data and the known values; and

determining physical and nutritional data including calories burned based on the type and duration of the physical activity performed.

4. The system of claim 3, in which the known values for physical activities are defined according to a series of Eigen activity values for comparison by the one or more programs for determining a physical activity of a subject wearing the device.

5. The system of claim 1, in which the one or more computer programs further comprise a program for distinguishing data associated with food and drink consumption from other data obtained from one or both of the one or more physiological sensors and/or a program for distinguishing data associated with physical activity from other data obtained from the device.

6. The system of claim 1, further comprising a database comprising dietary information for different foods, and a program for determining nutritional information from food dimension data obtained by the computer program for determining a dimension of a food item in image data obtained from the video camera.

7. The system of claim 1, wherein the device comprises a global positioning sensor and the system comprises a database comprising environmental quality information for different locations and a computer program for estimating a subject'"'"'s environmental exposure based on location data obtained from a GPS sensor on the device, weather data obtained from environmental monitoring sensors on the device, and the database of environmental quality information.

8. The system according to claim 1, in which the sensors connected to the bus include an accelerometer, a gyroscope, a GPS sensor, an oximeter, and a UV sensor.

9. The system of claim 8, in which the recording activity of the device is controlled by the output of the accelerometer, gyroscope, GPS sensor, oximeter, or UV sensor.

21. A method of monitoring behavior, condition or activity of a subject comprising, using the system of claim 1:

a) recording output of the device affixed to the subject in a data storage device in a computer comprising a processor connected to a data storage medium using executable instructions executed by the processor; and

b) processing, using executable instructions to be executed by a processor output of the device, to produce an output relating to a behavior or activity of the subject.

22. The method of claim 21, comprising:

a) recording image data from the two or more imaging sensors suitably positioned on a subject to record a stereo image pair of food consumed by the subject and an image of the physical activity of the subject; and

b) determining from the image data one or both of food consumption by the subject and physical activity of the subject.

23. The method of claim 21, in which the device comprises two cameras, an accelerometer, a gyroscope, a heart rate monitor, an oximeter, and a global positioning system (GPS) sensor and in which the subject is a police officer and the data recorded includes environmental data and physiological data.

24. The method of claim 21, in which the device comprises a GPS sensor, a heart rate monitor, and an accelerometer, and in which the subject is a consumer in a shopping environment and the data recorded includes geographical and physiological data.

25. The method of claim 21, in which the device comprises two cameras, a GPS sensor, a heart rate monitor, an accelerometer, and an oximeter and in which the subject is livestock and the data recorded includes geographical and physiological data.

26. The method of claim 21, in which the device comprises two cameras, an accelerometer, a gyroscope, a GPS sensor, an oximeter, and a UV sensor.

10. The system of claim i, in which the recording activity of the device is controlled by output of one or more of the sensors.

11. A device configured to be worn on a subject for the remote monitoring of food intake and physical activity in a subject, comprising:

a) a housing;

b) a bus inside the housing;

c) one or more sensors connected to the bus;

d) two or more imaging sensors connected to the bus and positioned in the housing to record a stereo image;

e) an eating detector in the housing and connected to the bus, eating detector comprising a light beam source configured in the device to pass a light beam in front of a subject'"'"'s mouth when the device is worn by a subject and a light sensor able to measure changes of light intensity in a path of a light beam from the light beam source;

f) a microprocessor connected to the bus;

g) a data storage device connected to the bus and comprising a data storage medium;

h) a program for managing one or more operations of the device including imaging, data acquisition and storage; and

i) a power supply connected to the bus.

12. The device of claim 11, in which data stored on the data storage device is encrypted.

13. The device of claim 11, in which the data storage device comprises a removable flash memory card.

14. The device of claim 11, in which the device further comprises a data transmission interface connected to the bus.

15. The device of claim 14, comprising a program for receiving data from an external computer and comparing the received data with data obtained from the sensors.

16. The device of claim 14, in which the data transmission interface operates under WiFi, cellular network or Bluetooth protocols.

17. The device of claim 14, in which the data transmission interface comprises a wired data communications interface.

18. The device of claim 11, wherein the one or more sensors are chosen from:

an accelerometer, a gyroscope, an ultraviolet sensor, an oxygen saturation sensor, an optical sensor, a heart rate monitor, an electrode, a global positioning sensor, a magnetometer, a temperature sensor, an oximeter, an accelerometer, a three-axis accelerometer, a gyroscope, a three-axis gyroscope, an audio sensor/player, and an environmental sensor, wherein the environmental sensor is chosen from one or more of an ultraviolet (UV) light sensor, a thermometer, a humidity sensor, a VOC sensor, a digital compass, and an ambient sound recorder.

19. The device of claim 11, in which output from the light sensor of the eating detector controls recording from the video cameras by either modifying a frame rate of the video cameras or turning the video cameras off or on.

20. The device of claim 11, in which the light beam source of the eating detector is a laser.