Detecting physiological responses while taking into account consumption of confounding substances

• US 20180092542A1
• Filed: 12/06/2017
• Published: 04/05/2018
• Est. Priority Date: 06/14/2015
• Status: Active Grant

• Current Assignee: Facense Ltd.
• Original Assignee: Facense Ltd.
• Inventors: Tzvieli, Arie, Thieberger, Gil, Frank, Ari M.
• Granted Patent: US 10,130,261 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2562/0271 Thermal or temperature sensors
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/168 Evaluating attention deficit, hyperactivity
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/486 Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/6814 Head
• CPC Class Codes: G01J 5/0025 Living bodies ear thermometers G01J5/0011; detecting, measuring or recording for diagnostic purposes A61B5/00
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Described herein are systems and methods for detecting a physiological response based on thermal measurements while accounting for consumption of a confounding substance such as a medication, alcohol, caffeine, or nicotine. In one embodiment, a system includes a computer and an inward-facing head-mounted thermal camera (CAM) that takes thermal measurements of a region of interest (TH_ROI) on a user'"'"'s face. The computer receives an indication indicative of the user consuming a confounding substance that affects TH_ROI, and detects the physiological response while the consumed confounding substance affects TH_ROI. The detection is based on TH_ROI, the indication and a model. Optionally, the model was trained on: a first set of TH_ROI taken while the confounding substance affected TH_ROI, and a second set of TH_ROI taken while the confounding substance did not affect TH_ROI.

Claims

1. A system configured to detect a physiological response while taking into account consumption of a confounding substance, comprising:

an inward-facing head-mounted thermal camera (CAM) configured to take thermal measurements of a region of interest (TH_ROI) on a user'"'"'s face; and

a computer configured to;

receive an indication indicative of the user consuming a confounding substance that affects TH_ROI; and

detect the physiological response, while the consumed confounding substance affects TH_ROI, based on TH_ROI, the indication, and a model.

2. The system of claim 1, wherein the model was trained on:

a first set of TH_ROI taken while the confounding substance affected TH_ROI, and a second set of TH_ROI taken while the confounding substance did not affect TH_ROI.

3. The system of claim 2, wherein CAM is located less than 10 cm from the user'"'"'s face, and the confounding substance comprises at least one of an alcoholic beverage, a medication, and a cigarette;

and wherein the first and second sets are previous TH_ROI of the user.

4. The system of claim 2, wherein the computer is configured to detect the physiological response based on first TH_ROI, for which there is no indication indicating that the first TH_ROI were affected by a consumption of the confounding substance, and the first TH_ROI reach a threshold;

and the computer is configured not to detect the physiological response based on second TH_ROI, for which there is an indication indicating that the second TH_ROI were affected by a consumption the confounding substance, and the second TH_ROI also reach the threshold.

5. The system of claim 2, wherein the model is a machine learning-based model, and the computer is further configured to generate feature values based on TH_ROI and the indication, and to utilize the machine learning-based model to detect the physiological response based on the feature values.

6. The system of claim 2, wherein the confounding substance comprises a medication, the indication is received from at least one of a pill dispenser and a sensor-enabled pill, and the indication indicates that the user took a medication.

7. The system of claim 2, wherein the confounding substance comprises an alcoholic beverage, the indication is received from at least one of a refrigerator, a pantry, and a serving robot;

and wherein the indication indicates that the user took an alcoholic beverage.

8. The system of claim 2, wherein the indication is received from a head-mounted visible-light camera having in its field of view a volume that protrudes out of the user'"'"'s mouth, and the computer is further configured to identify the consuming of the confounding substance based on analysis of images taken by the head-mounted visible-light camera.

9. The system of claim 2, wherein the indication is received from a device having an internet-of-things (IoT) capability through which the indication is provided to the computer.

10. The system of claim 2, wherein the indication is received from a microphone configured to record the user;

and the computer is further configured to identify the consuming of the confounding substance utilizing a sound recognition algorithm operated on a recording of the user.

11. The system of claim 10, wherein the sound recognition algorithm comprises a speech recognition algorithm configured to identify words that are indicative of consuming the confounding substance.

12. The system of claim 2, wherein the indication is received from a user interface configured to receive an input from the user or a third party about the consuming of the confounding substance.

13. The system of claim 2, wherein the region of interest (ROI) is on the forehead, and CAM:

is physically coupled to an eyeglasses frame, located below the ROI, and does not occlude the ROI.

14. The system of claim 2, wherein the region of interest (ROI) is on the periorbital area, and CAM is located less than 10 cm from the face.

15. The system of claim 2, wherein the region of interest (ROI) is on the nose, and CAM:

is physically coupled to an eyeglasses frame, and located less than 10 cm from the face.

16. The system of claim 2, wherein the region of interest (ROI) is below the nostrils, and CAM:

is physically coupled to an eyeglasses frame, located above the ROI, and does not occlude the ROI.

17. A method for detecting a physiological response while taking into account consumption of a confounding substance, comprising:

taking, using an inward-facing head-mounted thermal camera, thermal measurements of a region of interest (TH_ROI) on a user'"'"'s face;

receiving an indication indicative of consuming a confounding substance that affects TH_ROI; and

detecting the physiological response, while the consumed confounding substance affects TH_ROI, based on TH_ROI, the indication, and a model;

wherein the model was trained on;

a first set of TH_ROI taken while the confounding substance affected TH_ROI, and a second set of TH_ROI taken while the confounding substance did not affect TH_ROI.

18. The method of claim 17, wherein the model is a machine learning-based model, and further comprising generating feature values based on TH_ROI and the indication, and utilizing the machine learning-based model to detect the physiological response based on the feature values.

19. The method of claim 17, further comprising generating the indication based on analyzing images taken by a head-mounted visible-light camera having in its field of view at least a portion of a volume that protrudes out of the user'"'"'s mouth.

20. The method of claim 17, wherein the confounding substance comprises at least one of:

(i) a medication, wherein the indication is received from at least one of a pill dispenser and a sensor-enabled pill, and (ii) an alcoholic beverage, wherein the indication is received from at least one of a refrigerator, a pantry, and a serving robot.