Systems for collecting thermal measurements of the face

• US 10,113,913 B2
• Filed: 10/03/2016
• Issued: 10/30/2018
• Est. Priority Date: 10/03/2015
• Status: Active Grant

• Current Assignee: Facense Ltd.
• Original Assignee: Facense Ltd.
• Inventors: Tzvieli, Arie, Frank, Ari M, Thieberger, Gil
• Primary Examiner(s): Lamprecht, Joel
• Application Number:
  US15/284,528
• Publication Number:
  US 20170095157A1
• Time in Patent Office: 757 Days
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/02438 with portable devices, e.g. worn by the patient
• CPC Class Codes: A61B 5/08 Detecting, measuring or recording devices for evaluating the respiratory organs A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/164 Lie detection
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/411 Detecting or monitoring allergy or intolerance reactions to an allergenic agent or substance
• CPC Class Codes: A61B 5/4824 Touch or pain perception evaluation A61B5/4821 takes precedence
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/6814 Head
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: A61B 5/7278 Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: G01J 5/0265 Handheld, portable ear thermometers G01J5/049
• CPC Class Codes: G01J 5/12 using thermoelectric elements, e.g. thermocouples
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 21/32 : using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

This disclosure describes various systems for collecting thermal measurements of regions of a user'"'"'s face. Each of the systems includes a frame configured to be worn on the user'"'"'s head, and one or more lightweight thermal cameras that are coupled to the frame and configured to take thermal measurements of a region of interest on the user'"'"'s face. Due to their coupling to the frame, the thermal cameras remain pointed at their respective regions of interest even when the user'"'"'s head performs angular movements. The thermal measurements collected by some embodiments of the systems described herein may be utilized for a variety of applications that involve detecting different types of physiological responses or medical disorders.

Claims

1. A system configured to take thermal measurements indicative of a physiological response, comprising:

a frame configured to be worn on a user'"'"'s head;

first, second, third, and fourth thermal cameras, which are physically coupled to the frame;

wherein the frame is configured to hold the first, second, third, and fourth thermal cameras less than 15 cm away from the user'"'"'s face;

the first and third thermal cameras are located to the right of the vertical symmetry axis that divides the face;

the second and fourth thermal cameras are located to the left of the vertical symmetry axis; and

the third and fourth thermal cameras are located at least 1 cm below the first and second thermal cameras, respectively;

the first thermal camera is configured to take thermal measurements of a first region of interest (TH_ROI1), wherein ROI₁ covers a portion of the right side of the user'"'"'s forehead;

the second thermal camera is configured to take thermal measurements of a second ROI (TH_ROI2), wherein ROI₂ covers a portion of the left side of the forehead;

the third thermal camera is configured to take thermal measurements of a third ROI (THR_OI3), wherein ROI₃ covers a portion of the right side of the user'"'"'s upper lip; and

the fourth thermal camera is configured to take thermal measurements of a fourth ROI (TH_ROI4), wherein ROI₄ covers a portion of the left side of the user'"'"'s upper lip.

2. The system of claim 1, wherein each of the first, second, third, and fourth thermal cameras weighs below 5 g, and further comprising a processor configured to detect the physiological response based on TH_ROI1, TH_ROI2, TH_ROI3, and TH_ROI4.

3. The system of claim 2, wherein the physiological response is indicative of stress felt by the user.

4. The system of claim 2, wherein the physiological response is indicative of an allergic reaction of the user.

5. The system of claim 2, wherein the physiological response is indicative of a level of pain felt by the user.

6. The system of claim 2, wherein the physiological response is indicative of an occurrence of at least one of the following emotional states of the user:

fear, anxiety, guilt, pain, and sexual arousal.

7. The system of claim 2, wherein the overlap between ROI₁ and ROI₂ is lower than 50% of the smallest area from among the areas of ROI₁ and ROI₂, and the overlap between ROI₃ and ROI₄ is lower than 50% of the smallest area from among the areas of ROI₃ and ROI₄.

8. The system of claim 2, wherein there is no overlap between ROI₁ and ROI₂, and there is no overlap between ROI₃ and ROI₄.

9. The system of claim 2, wherein the processor is further configured to compare one or more values derived from TH_ROI1, TH_ROI2, TH_ROI3, and TH_ROI4 to a threshold, and to determine whether the threshold is reached;

whereby the one or more values reaching the threshold is indicative that the user had the physiological response.

10. The system of claim 2, wherein the processor is further configured to calculate a value indicative of a similarity between a reference time series corresponding to the physiological response and a time series based on TH_ROI1, TH_ROI2, TH_ROI3, and TH_ROI4;

whereby the similarity reaching a threshold is indicative that the user had the physiological response.

11. The system of claim 2, wherein the processor is further configured to generate feature values based on TH_ROI1, TH_ROI2 TH_ROI3, and TH_ROI4, and to utilize a machine learning-based model to calculate, based on the feature values, a value indicative of whether the user had the physiological response.

12. The system of claim 1, wherein the third and fourth thermal cameras are located outside the exhale streams of the mouth and nostrils, and the first, second, third and fourth thermal cameras are located less than 5 cm away from the face.

13. The system of claim 1, further comprising a fifth thermal camera coupled to the frame, pointed at a fifth ROI (ROI₅), wherein ROI₅ covers a portion of the user'"'"'s nose.

14. The system of claim 1, further comprising a fifth thermal camera coupled to the frame, pointed at a fifth ROI (ROI₅), wherein ROI₅ covers a portion of periorbital region of the face.

15. The system of claim 1, wherein each of the first, second, third and fourth thermal cameras is based on a thermopile sensor or a microbolometer sensor, and remains pointed at its respective ROI when the head makes angular movements above 0.1 rad/sec.

16. A method for detecting a physiological response, comprising:

receiving, by a processor, thermal measurements from a system configured to be worn on a user'"'"'s head;

wherein the system comprises first, second, third, and fourth thermal cameras;

the first and third thermal cameras are located to the right of the vertical symmetry axis that divides the face;

the second and fourth thermal cameras are located to the left of the vertical symmetry axis; and

the third and fourth thermal cameras are located at least 1 cm below the first and second thermal cameras, respectively;

the first thermal camera is configured to take thermal measurements of a first region of interest (TH_ROI1), wherein ROI₁ covers a portion of the right side of the user'"'"'s forehead;

the second thermal camera is configured to take thermal measurements of a second ROI (TH_ROI2), wherein ROI₂ covers a portion of the left side of the forehead;

the third thermal camera is configured to take thermal measurements of a third ROI (TH_ROI3), wherein ROI₃ covers a portion of the right side of the user'"'"'s upper lip;

the fourth thermal camera is configured to take thermal measurements of a fourth ROI (TH_ROI4), wherein ROI₄ covers a portion of the left side of the user'"'"'s upper lip;

generating feature values based on TH_ROI1, TH_ROI2, TH_ROI3, and TH_ROI4; and

utilizing, by the processor, a machine learning-based model for calculating, based on the feature values, a value indicative of whether the user had the physiological response.

17. The method of claim 16, wherein calculating the value indicative of whether the user had the physiological response comprises calculating a value indicative of at least one of the following:

stress felt by the user, allergic reaction of the user, pain felt by the user, fear felt by the user, anxiety felt by the user, and sexual arousal felt by the user.

18. A method for detecting a physiological response, comprising:

receiving, by a processor, thermal measurements from a system configured to be worn on a user'"'"'s head;

wherein the system comprises first, second, third, and fourth thermal cameras;

the first and third thermal cameras are located to the right of the vertical symmetry axis that divides the face;

the second and fourth thermal cameras are located to the left of the vertical symmetry axis; and

the third and fourth thermal cameras are located at least 1 cm below the first and second thermal cameras, respectively;

the first thermal camera is configured to take thermal measurements of a first region of interest (TH_ROI1), wherein ROI₁ covers a portion of the right side of the user'"'"'s forehead;

the second thermal camera is configured to take thermal measurements of a second ROI (TH_ROI2), wherein ROI₂ covers a portion of the left side of the forehead;

the third thermal camera is configured to take thermal measurements of a third ROI (TH_ROI3), wherein ROI₃ covers a portion of the right side of the user'"'"'s upper lip;

the fourth thermal camera is configured to take thermal measurements of a fourth ROI (TH_ROI4), wherein ROI₄ covers a portion of the left side of the user'"'"'s upper lip;

comparing, by the processor, one or more values derived from TH_ROI1, TH_ROI2, TH_ROI3, and TH_ROI4 to a threshold; and

responsive to the one or more values reaching the threshold, indicating that the user had the physiological response.

19. The method of claim 18, wherein comparing the one or more values to the threshold comprises:

(i) calculating a value indicative of a similarity between a reference time series corresponding to the physiological response and a time series based on TH_ROI1, TH_ROI2, TH_ROI3, and TH_ROI4; and

(ii) comparing the value indicative of the similarity to the threshold;

whereby an incident, in which the value indicative of the similarity reaches the threshold, is indicative that the user had the physiological response.

20. The method of claim 18, wherein indicating that the user had the physiological response comprises indicating at least one of the following:

stress felt by the user, an allergic reaction of the user, pain felt by the user, fear felt by the user, anxiety felt by the user, and sexual arousal felt by the user.