Wearable respiration measurements system

• US 10,136,856 B2
• Filed: 06/27/2017
• Issued: 11/27/2018
• Est. Priority Date: 06/27/2016
• Status: Active Grant

• Current Assignee: Facense Ltd.
• Original Assignee: Facense Ltd.
• Inventors: Tzvieli, Arie, Thieberger, Gil, Frank, Ari M
• Primary Examiner(s): Porta, David
• Assistant Examiner(s): BOOSALIS, FANI POLYZOS
• Application Number:
  US15/635,178
• Publication Number:
  US 20170367651A1
• Time in Patent Office: 518 Days
• CPC Class Codes: A61B 2503/12 Healthy persons not otherwise provided for, e.g. subjects of a marketing survey
• CPC Class Codes: A61B 2562/0204 Acoustic sensors
• CPC Class Codes: A61B 2562/0233 Special features of optical sensors or probes classified in A61B5/00
• CPC Class Codes: A61B 2562/0271 Thermal or temperature sensors
• CPC Class Codes: A61B 2562/046 in a matrix array
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/0823 Detecting or evaluating cough events
• CPC Class Codes: A61B 5/0878 using temperature sensing means
• CPC Class Codes: A61B 5/091 Measuring volume of inspired or expired gases, e.g. to determine lung capacity
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/486 Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: A61B 5/7275 Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
• CPC Class Codes: A61B 5/7282 : Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/746 : Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: A61B 5/7475 : User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: A61B 7/003 : Detecting lung or respiration noise
• CPC Class Codes: A61M 2021/0088 : modulated by a simulated respiratory frequency
• CPC Class Codes: A61M 21/02 : for inducing sleep or relaxation, e.g. by direct nerve stimulation, hypnosis, analgesia for massage A61H; electrotherapy A61N, e.g. applying alternating or intermittent electric currents for producing anaesthesia A61N1/36021
• CPC Class Codes: A61M 2205/507 : Head Mounted Displays [HMD]
• CPC Class Codes: A61M 2230/42 : Rate
• CPC Class Codes: A61M 2230/50 : Temperature
• CPC Class Codes: A61M 2230/63 : Motion, e.g. physical activity
• CPC Class Codes: A61M 2230/65 : Impedance, e.g. conductivity, capacity
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A wearable system configured to collect thermal measurements related to respiration. The system includes a frame configured to be worn on a user'"'"'s head, and at least one non-contact thermal camera (e.g., thermopile or microbolometer based sensor). The thermal camera is small and lightweight, physically coupled to the frame, located close to the user'"'"'s face, does not occlude any of the user'"'"'s mouth and nostrils, and is configured to take thermal measurements of: a portion of the right side of the user'"'"'s upper lip, a portion of the left side of the user'"'"'s upper lip, and a portion of the user'"'"'s mouth. The thermal measurements are forwarded to a computer that calculates breathing related parameters, such as breathing rate, an extent to which the breathing was done through the mouth, an extent to which the breathing was done through the nostrils, and ratio between exhaling and inhaling durations.

Claims

1. A system configured to collect thermal measurements related to respiration, comprising:

a frame configured to be worn on a user'"'"'s head; and

at least one thermal camera that is physically coupled to the frame;

wherein the frame is configured to hold the at least one thermal camera less than 15 cm away from the user'"'"'s face, such that the at least one thermal camera does not occlude any of the user'"'"'s mouth and upper lip;

wherein the at least one thermal camera is configured to take thermal measurements of;

a portion of the right side of the user'"'"'s upper lip (TH_ROI1), a portion of the left side of the user'"'"'s upper lip (TH_ROI2), and a portion of the user'"'"'s mouth (TH_ROI3).

2. The system of claim 1, wherein the at least one thermal camera weighs below 5 g, and TH_ROI1, TH_ROI2, and TH_ROI3 are indicative of the breathing rate through at least one of the mouth and nostrils;

and further comprising a computer configured to calculate the breathing rate based on TH_ROI1, TH_ROI2, and TH_ROI3.

3. The system of claim 1, further comprising a computer configured to calculate, based on TH_ROI1, TH_ROI2, and TH_ROI3, a value indicative of at least one of the following values:

an extent to which the user'"'"'s breathing was done through the mouth, and an extent to which the user'"'"'s breathing was done through the nostrils.

4. The system of claim 1, wherein the at least one thermal camera is based on microbolometer sensing elements or thermopile sensing elements, and comprises a focal-plane array (FPA) sensor that provides at least TH_ROI1 and TH_ROI3.

5. The system of claim 1, wherein the at least one thermal camera comprises at least first and second thermal cameras, configured to be located at least 0.5 cm to the right and to the left of the vertical symmetry axis that divides the face, respectively, and above the tip of the user'"'"'s nose;

and wherein the first thermal camera is configured to take TH_ROI1, and the second thermal camera is configured to take TH_ROI2.

6. The system of claim 1, wherein ROI₁ covers more than 60% of a 2×

2 cm area adjacent to the right side of the vertical symmetry axis that divides the face and below the right nostril of the user, ROI₂ covers more than 60% of a 2×

2 cm area adjacent to the left side of the vertical symmetry axis and below the left nostril of the user, and ROI₃ covers more than 60% of a 2×

5 cm area located over the vertical symmetry axis and below the nostrils of the user.

7. The system of claim 1, further comprising an in-the-ear earbud comprising a microphone, configured to measure breathing sounds inside the ear canal of the user;

and further comprising a computer is configured to identify inhales based on audio signal analysis of the recordings from the earbud and TH_ROI1, TH_ROI2, and TH_ROI3.

8. The system of claim 7, wherein the computer is further configured to train a machine learning model to identify the inhales based on feature values generated based on data comprising TH_ROI1, TH_ROI2, TH_ROI3, and labels comprising identifications of the inhales that were calculated based on the audio signal analysis of the recordings from the earbud.

9. The system of claim 1, further comprising a computer configured to calculate a ratio between mouth breathing and nasal breathing based on TH_ROI1, TH_ROI2, and TH_ROI3;

and the system is further configured to help the user to prefer nasal breathing over mouth breathing by notifying the user when the ratio between the mouth breathing and the nasal breathing reaches a predetermined threshold.

10. The system of claim 1, further comprising a computer configured to calculate, based on TH_ROI1 and TH_ROI2, a ratio between an extent the user breathed through the right nostril and an extent the user breathed through the left nostril.

11. The system of claim 10, wherein the system is further configured to help the user to develop awareness to the dominant nostril by at least one of the following:

notifying the user when the ratio is balanced, notifying the user when the ratio reaches a predetermined threshold, notifying the user when a trend in values of the ratio changes direction, notifying the user when the right nostril is the dominant nostril, and notifying the user when the left nostril is the dominant nostril.

12. The system of claim 1, further comprising a computer configured to (i) receive an indication that the user suffers from asthma, (ii) identify, based on TH_ROI1, TH_ROI2, and TH_ROI3, an increase in the user'"'"'s breathing rate above a predetermined threshold, and (iii) alert the user, via a user interface, about an imminent asthma attack.

13. The system of claim 12, further comprising a microphone configured to record the user;

the computer is further configured to analyze the recording in order to identify at least one of the following body sounds;

asthmatic breathing sounds, asthma wheezing, and coughing;

wherein a first alert provided to the user in response to identifying the increase in the user'"'"'s breathing rate above the predetermined threshold without identifying at least one of the body sounds is less intense than a second alert provided to the user in response to identifying both the increase in the user'"'"'s breathing rate above the predetermined threshold and at least one of the body sounds.

14. The system of claim 1, further comprising a computer configured to calculate, based on TH_ROI1, TH_ROI2, and TH_ROI3, the ratio between exhale and inhale durations (t_exhale/t_inhale), and to alert the user, via a user interface, to increase the ratio responsive to detecting that t_exhale/t_inhale is below a predetermined threshold.

15. The system of claim 1, further comprising a computer configured to:

receive a first indication that the user'"'"'s stress level reaches a threshold, detect at least one of the following two conditions based on TH_ROI1, TH_ROI2 and TH_ROI3;

(i) that the ratio between exhaling and inhaling durations is below 1.5 (t_exhale/t_inhale<

1.5), and (ii) that the user'"'"'s breathing rate reached a predetermined threshold, and command a user interface to suggest the user to increase t_exhale/t_inhale to at least 1.5.

16. The system of claim 15, wherein the computer is further configured to command the user interface to:

update the user about changes in the stress level in response to increasing t_exhale/t_inhale, and provide positive reinforcement to help the user to maintain the required ratio at least until a certain improvement in the stress level is achieved.

17. The system of claim 1, further comprising a computer configured to:

receive a first indication that the user is about to have an imminent event, detect at least one of the following two conditions based on TH_ROI1, TH_ROI2 and TH_ROI3;

(i) that the ratio between exhaling and inhaling durations is below 1.5 (t_exhale/t_inhale<

1.5), and (ii) that the user'"'"'s breathing rate reached a predetermined threshold, and command a user interface to suggest the user to increase t_exhale/t_inhale.

18. The system of claim 17, wherein the imminent event comprises at least one of the following events:

entering a social interaction, entering a meeting, entering one home, entering a pleasant or unpleasant situation, entering a place with unpleasant temperature, taking a test, starting a game session, making a speech, making a phone call, responding to a request to make a decision, reading something that is expected to affect the user'"'"'s emotions above a predetermined threshold, and reaching a traffic jam.

19. The system of claim 1, further comprising a computer configured to:

receive a first indication that the user is going to make a physical effort, command a user interface to suggest the user to exhale while making the physical effort, and determine based on TH_ROI1, TH_ROI2, and TH_ROI3 whether the user exhaled while making the physical effort.

20. The system of claim 19, wherein the computer is further configured to command the user interface to play a positive feedback in response to determining that the user managed to exhale while making the physical effort;

wherein the physical effort includes at least one of the following activities;

standing up, sitting down, manipulating with the hands an item that requires applying a significant force, defecating, dressing, leaning over, and lifting an item.

21. The system of claim 1, further comprising a computer configured to calculate, based on TH_ROI1, TH_ROI2, and TH_ROI3, at least one of:

lung volumes, and tidal volume.