Detecting an allergic reaction from nasal temperatures

• US 10,136,852 B2
• Filed: 12/06/2017
• Issued: 11/27/2018
• Est. Priority Date: 06/14/2015
• Status: Active Grant

• Current Assignee: Facense Ltd.
• Original Assignee: Facense Ltd.
• Inventors: Tzvieli, Arie, Thieberger, Gil, Frank, Ari M
• Primary Examiner(s): Porta, David
• Assistant Examiner(s): BOOSALIS, FANI POLYZOS
• Application Number:
  US15/832,815
• Publication Number:
  US 20180092592A1
• Time in Patent Office: 356 Days
• CPC Class Codes: A61B 2562/0271 Thermal or temperature sensors
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/316 Modalities, i.e. specific diagnostic methods
• CPC Class Codes: A61B 5/411 Detecting or monitoring allergy or intolerance reactions to an allergenic agent or substance
• CPC Class Codes: A61B 5/4884 inducing physiological or psychological stress, e.g. applications for stress testing
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/6814 Head
• CPC Class Codes: A61B 5/6819 Nose
• CPC Class Codes: G01J 5/0025 Living bodies ear thermometers G01J5/0011; detecting, measuring or recording for diagnostic purposes A61B5/00
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Described herein are systems and methods for personalized detection of an allergic reaction of a user. In one embodiment, a system includes an inward-facing head-mounted thermal camera (CAM) and a computer. CAM takes thermal measurements of a region on the nose (TH_N) of a user. The computer generates feature values based on TH_N and utilizes a model to detect an allergic reaction of the user based on the feature values. The model is trained based on previous TH_N of the user taken during different days. Some embodiments may utilize additional thermal cameras that take thermal measurements of other regions on the face, which may be utilized by the computer to generate additional feature values that are used to detect the allergic reaction. In some cases, detection of the allergic reaction may occur following a rise in nasal temperature, even before the user becomes aware of the allergic reaction.

Claims

1. A system configured to detect allergic reaction, comprising:

an inward-facing head-mounted thermal camera (CAM) configured to take thermal measurements of a region on the nose (TH_N) of a user; and

a computer configured to;

generate feature values based on TH_N, and utilize a machine learning-based model to detect an allergic reaction of the user based on the feature values;

wherein the machine learning-based model was trained based on previous TH_N taken during different days.

2. The system of claim 1, wherein the previous TH_N are previous TH_N of the user, and further comprising a frame configured to be worn on the user'"'"'s head;

wherein CAM is physically coupled to the frame and the frame is configured to hold CAM less than 15 cm from the user'"'"'s face.

3. The system of claim 2, further comprising second and third inward-facing head-mounted thermal cameras (CAM2 and CAM3, respectively), each of which is physically coupled to the frame;

wherein the frame is configured to hold CAM2 and CAM3 less than 15 cm from the user'"'"'s face, to the right and to the left of the vertical symmetry axis that divides the user'"'"'s face, respectively;

wherein CAM2 and CAM3 are configured to take thermal measurements of regions on the right and left cheeks (TH_ROI1 and TH_ROI2, respectively); and

wherein the computer is further configured to generate additional feature values based on TH_ROI1 and TH_ROI2, and to utilize the additional feature values to detect the allergic reaction.

4. The system of claim 2, further comprising second and third inward-facing head-mounted thermal cameras (CAM2 and CAM3, respectively), each of which is physically coupled to the frame;

wherein the frame is configured to hold CAM2 and CAM3 less than 15 cm from the user'"'"'s face, to the right and to the left of the vertical symmetry axis that divides the user'"'"'s face, respectively;

wherein CAM2 and CAM3 are configured to take thermal measurements of regions on the right and left periorbital areas (TH_ROI1 and TH_ROI2, respectively); and

wherein the computer is further configured to generate additional feature values based on TH_ROI1 and TH_ROI2, and to utilize the additional feature values to detect the allergic reaction.

5. The system of claim 2, wherein the computer is further configured to detect an early rise in the temperature of the region of the nose, which is evident before the user is aware of a symptom of the allergic reaction;

and further comprising a user interface configured to alert the user of a possible allergic reaction before the user is aware of the symptom of the allergic reaction, responsive to a detection of the early rise in the temperature of the region of the nose.

6. The system of claim 1, further comprising a sensor configured to take additional measurements (m_conf) of at least one of the following:

the user, and the environment in which the user was in while TH_N were taken;

wherein the computer is further configured to generate, based on m_conf, one or more feature values and to utilize the one or more feature values to detect the allergic reaction;

whereby m_conf are indicative of an extent to which a confounding factor occurred while TH_N were taken.

7. The system of claim 1, wherein the previous TH_N comprise:

a first set of TH_N taken while the user had an allergic reaction, and a second set of TH_N taken while the user did not have an allergic reaction.

8. The system of claim 1, wherein the computer is further configured to generate samples based on data comprising the previous TH_N and corresponding indications of whether the user experienced an allergic reaction, and to train the machine learning-based model based on the samples;

and wherein the previous TH_N comprise;

a first set of TH_N taken while the user was indoors, and a second set of TH_N taken while the user was outdoors.

9. The system of claim 1, wherein the computer is further configured to generate samples based on data comprising the previous TH_N and corresponding indications of whether the user experienced an allergic reaction, and to train the machine learning-based model based on the samples;

and wherein the previous TH_N comprise;

a first set of TH_N taken while the user was sitting, and a second set of TH_N taken while the user was walking.

10. The system of claim 1, wherein the computer is further configured to:

(i) generate samples based on data comprising the previous TH_N, corresponding indications of whether the user experienced an allergic reaction, and additional physiological measurements taken while the previous TH_N were taken, and (ii) train the machine learning-based model based on the samples; and

wherein the additional physiological measurements are indicative of one or more of the following signals of the user;

heart rate, heart rate variability, brainwave activity, galvanic skin response, muscle activity, and extent of movement.

11. The system of claim 1, wherein the computer is further configured to:

(i) generate samples based on data comprising the previous TH_N, corresponding indications of whether the user experienced an allergic reaction, and measurements of the environment in which the user was in while the previous TH_N were taken, and (ii) train the machine learning-based model based on the samples; and

wherein the measurements of the environment are indicative of one or more of the following values of the environment;

temperature, humidity level, noise level, air quality, wind speed, and infrared radiation level.

12. The system of claim 1, wherein the computer is further configured to:

(i) generate samples based on data comprising the previous TH_N, corresponding indications of whether the user experienced an allergic reaction, and indications of exposure of the user to one or more allergens up to twelve hours before some of the previous TH_N were taken, and (ii) train the machine learning-based model based on the samples;

wherein the indications of exposure are indicative of one or more of the following;

consumption of a certain drug, consumption of a certain food item, exposure to at least a certain concentration of pollen, exposure to at least a certain concentration of dust, exposure to a certain cosmetics product, and exposure to a certain aerosol.

13. A method for performing a personalized detection of an allergic reaction, comprising:

taking thermal measurements of a region on the nose (TH_N) of a user utilizing an inward-facing head-mounted thermal camera;

generating feature values based on TH_N; and

utilizing a machine learning-based model for detecting an allergic reaction of the user based on the feature values;

wherein the machine learning-based model was trained based on previous TH_N taken during different days.

14. The method of claim 13, wherein the previous TH_N comprise:

a first set of TH_N taken while the user had an allergic reaction, and a second set of TH_N taken while the user did not have an allergic reaction; and

further comprising detecting an early rise in the temperature on the region of the nose, which is evident before the user is aware of a symptom of the allergic reaction, and responsive to detecting the early rise in the temperature on the region of the nose, alerting the user about a possible allergic reaction before the user is aware of the symptom of the allergic reaction.

15. The method of claim 13, further comprising:

utilizing second and third inward-facing head-mounted thermal cameras (CAM2 and CAM3, respectively) for taking thermal measurements of regions on the right and left sides of the cheeks (TH_ROI1 and TH_ROI2, respectively);

generating additional feature values based on TH_ROI1 and TH_ROI2; and

detecting the allergic reaction based on the feature values and the additional feature values;

wherein CAM2 and CAM3 are physically coupled to a frame worn on the user'"'"'s head, which is configured to hold CAM2 and CAM3 less than 15 cm from the user'"'"'s face at positions that are to the right and to the left of the vertical symmetry axis that divides the user'"'"'s face, respectively.

16. The method of claim 13, further comprising:

utilizing second and third inward-facing head-mounted thermal cameras (CAM2 and CAM3, respectively) for taking thermal measurements of regions on the right and left periorbital areas (TH_ROI1 and TH_ROI2, respectively);

generating additional feature values based on TH_ROI1 and TH_ROI2; and

detecting the allergic reaction based on the feature values and the additional feature values;

wherein CAM2 and CAM3 are physically coupled to a frame worn on the user'"'"'s head, which is configured to hold CAM2 and CAM3 less than 15 cm from the user'"'"'s face at positions that are to the right and to the left of the vertical symmetry axis that divides the user'"'"'s face, respectively.

17. The method of claim 13, wherein the previous TH_N comprise:

a first set of TH_N taken while the user had an allergic reaction, and a second set of TH_N taken while the user did not have an allergic reaction.

18. The method of claim 13, further comprising:

receiving indications indicating whether the user experienced an allergic reaction, and training the machine learning-based model based on a first subset of the previous TH_N taken while the user was indoors, and a second subset of the previous TH_N taken while the user was outdoors.

19. The method of claim 13, further comprising:

receiving indications indicating whether the user experienced an allergic reaction, and training the machine learning-based model based on a first subset of the previous TH_N taken while the user was sitting, and a second subset of the previous TH_N taken while the user was walking.

20. The method of claim 13, further comprising (i) generating samples based on data comprising the previous TH_N, corresponding indications whether the user experienced an allergic reaction, and indications of exposure of the user to one or more allergens while some of the previous TH_N were taken, or up to twelve hours before the some of the previous TH_N were taken, and (ii) training the machine learning-based model based on the samples;

wherein the indications of exposure are indicative of one or more of the following;

consumption of a certain drug, consumption of a certain food item, exposure to at least a certain concentration of pollen, exposure to at least a certain concentration of dust, exposure to a certain cosmetics product, and exposure to a certain aerosol.