Breathing pattern identification for respiratory function assessment

• US 10,219,739 B2
• Filed: 11/25/2014
• Issued: 03/05/2019
• Est. Priority Date: 10/02/2013
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Mestha, Lalit K., Shilla, Eribaweimon, Bernal, Edgar A., Pennington, Graham Stephen, Madhu, Himanshu Jayantkumar
• Primary Examiner(s): Mallari, Patricia
• Assistant Examiner(s): Portillo, Jairo
• Application Number:
  US14/553,659
• Publication Number:
  US 20150094597A1
• Time in Patent Office: 1,561 Days
• Field of Search: None
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0013 Medical image data A61B1/00011, A61B6/56, A61B8/56 take precedence
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1073 Measuring volume, e.g. of limbs
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/113 occurring during breathing
• CPC Class Codes: A61B 5/4818 Sleep apnoea
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: A61B 5/747 in case of emergency, i.e. alerting emergency services

Abstract

What is disclosed is a system and method for identifying a patient'"'"'s breathing pattern for respiratory function assessment without contact and with a depth-capable imaging system. In one embodiment, a time-varying sequence of depth maps are received of a target region of a subject of interest over a period of inspiration and expiration. Once received, the depth maps are processed to obtain a breathing signal for the subject. The subject'"'"'s breathing signal comprises a temporal sequence of instantaneous volumes. One or more segments of the subject'"'"'s breathing signal are then compared against one or more reference breathing signals each associated with a known pattern of breathing. As a result of the comparison, a breathing pattern for the subject is identified. The identified breathing pattern is then used to assess the subject'"'"'s respiratory function. The teachings hereof find their uses in an array of diverse medical applications. Various embodiments are disclosed.

Claims

1. A method for determining that a subject is having an apnea episode, comprising:

capturing images of the subject while illuminated by a projected pattern of structured light, wherein the projected pattern of structured light is emitted by light emitting diodes arrayed in a uniform pattern fixed to an item of clothing worn by the subject;

generating a time-varying sequence of depth maps from the images based upon a comparison of spatial characteristics of reflections of the projected pattern of structured light in conjunction with a known distance between a light projector that projects the projected pattern of structured light and a camera that captures the images and characterized distortions at different locations to calculate a depth map for each one of the images;

receiving the time-varying sequence of depth maps obtained for a target region of the subject;

the use of a processor, the processor performing the operations of;

continuously analyzing the time-varying sequence of depth maps to determine a time duration of each time period when a change in the time-varying sequence of depth maps is at a minimum, wherein the continuously analyzing comprises extracting a volume from each one of the time-varying sequence of depth maps based on a tessellation of a target area of the subject formed by points in a three dimensional hull defined by three dimensional coordinates, wherein the tessellation comprises uniform or non-uniform geometric shapes that fill an entire surface of the three dimensional hull and do not overlap and computing differences in the volume extracted from the each one of the time-varying sequence of depth maps, wherein the differences in the volume is used to determine the time duration when the change in the time-varying sequence of depth maps is at the minimum;

determining that the subject is having the apnea episode in response to the time duration being at least a pre-defined length of time; and

sending a text message to a cellphone of a medical practitioner in response to the determining that the subject is having the apnea episode.

2. The method of claim 1, wherein the target region of the subject comprises at least one of:

an anterior thoracic region, a region of a dorsal body, a side view containing a thoracic region, or a view with a sheet covering a body of the subject.

3. The method of claim 1, wherein the camera comprises an image-based depth sensing device comprising any of:

a red green blue depth (RGBD) camera, an infrared depth camera, a passive stereo camera, an array of cameras, an active stereo camera, or a monocular video camera.

4. The method of claim 1, wherein the camera comprises any of:

a red green blue (RGB) camera, an infrared camera, a multispectral camera, or a hyperspectral camera.

5. The method of claim 1, wherein, in response to the subject being an infant, determining that the subject is at risk for sudden infant death syndrome.

6. The method of claim 1, wherein, in response to the subject having the apnea episode, performing any of:

initiating an alert, or signaling a medical professional.

7. The method of claim 1, further comprising:

communicating to any of;

a memory, a storage device, a display device, a handheld wireless device, a handheld cellular device, or a remote device over a network that the subject is having the apnea episode.

8. The method of claim 1, wherein the pre-defined length of time is at least 20 seconds.

9. The method of claim 1, wherein the determining that the subject is having the apnea episode occurs in real-time.

10. The method of claim 1, wherein the analyzing further comprises:

extracting shape descriptors from each of the time-varying sequence of depth maps, the shape descriptors being used for comparison purposes to determine the time duration when a change in shape is at a minimum.

11. A system for determining that a subject is having an apnea episode, comprising:

an array of light emitting diodes in a uniform pattern fixed to an item of clothing worn by the subject to project a pattern of structured light;

a camera to capture images of the subject while illuminated by the pattern of structured light and to generate a time-varying sequence of depth maps from the images based upon a comparison of spatial characteristics of reflections of the pattern of structured light in conjunction with a known distance between the light projector and the camera that captures the images and characterized distortions at different locations to calculate a depth map for each one of the images; and

a work station in communication with the camera over a network, the work station comprising;

a processor; and

a memory storing machine readable program instructions, which when executed by the processor cause the processor to perform operations, the operations comprising;

receiving the time-varying sequence of depth maps obtained for a target region of the subject;

continuously analyzing the time-varying sequence of depth maps to determine a time duration of each time period when a change in the time-varying sequence of depth maps is at a minimum, wherein the continuously analyzing comprises extracting a volume from each one of the time-varying sequence of depth maps based on a tessellation of a target area of the subject formed by points in a three dimensional hull defined by three dimensional coordinates, wherein the tessellation comprises uniform or non-uniform geometric shapes that fill an entire surface of the three dimensional hull and do not overlap and computing differences in the volume extracted from the each one of the time-varying sequence of depth maps, wherein the differences in the volume is used to determine the time duration when the change in the time-varying sequence of depth maps is at the minimum;

determining that the subject is having the apnea episode in response to the time duration being at least a pre-defined length of time; and

sending a text message to a cellphone of a medical practitioner in response to the determining that the subject is having the apnea episode.

12. The system of claim 11, wherein the target region of the subject comprises at least one of:

an anterior thoracic region, a region of a dorsal body, a side view containing a thoracic region, or a view with a sheet covering a body of the subject.

13. The system of claim 11, wherein the camera comprises an image-based depth sensing device comprising any of:

a red green blue depth (RGBD) camera, an infrared depth camera, a passive stereo camera, an array of cameras, an active stereo camera, or a monocular video camera.

14. The system of claim 11, wherein the camera comprises any of:

a red green blue (RGB) camera, an infrared camera, a multispectral camera, or a hyperspectral camera.

15. The system of claim 11, wherein the pre-defined length of time is at least 20 seconds.

16. The system of claim 11, wherein the analyzing further comprising:

extracting shape descriptors from each of the time-varying sequence of depth maps, the shape descriptors being used for comparison purposes to determine the time duration when a change in shape is at a minimum.