Processing a video for respiration rate estimation

• US 9,301,710 B2
• Filed: 06/21/2012
• Issued: 04/05/2016
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Mestha, Lalit Keshav, Bernal, Edgar A., Xu, Beilei
• Primary Examiner(s): Brutus, Joel F
• Application Number:
  US13/529,648
• Publication Number:
  US 20130324875A1
• Time in Patent Office: 1,384 Days
• Field of Search: 600/411
• US Class Current: 1/1
• CPC Class Codes: A61B 5/0064 Body surface scanning
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/1135 by monitoring thoracic expansion
• CPC Class Codes: G01B 11/2527 with phase change by in-plane movement of the patern
• CPC Class Codes: G06F 2218/10 by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/20044 Skeletonization; Medial axis transform
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G09G 2380/08 Biomedical applications

Abstract

What is disclosed is a system and method for estimating a respiration rate by analyzing distortions in reflections of structured illumination patterns captured in a video containing a view of a subject'"'"'s thoracic region. In one embodiment, a video of a target region of a body of a subject of interest is received. Video image frames are processed to estimate 3D time-series data for the target region. As more fully disclosed herein, the subject'"'"'s respiration rate is estimated from the 3D time-series data. Measurements can be acquired under a diverse set of lighting conditions. The teachings hereof provide a non-contact approach to patient respiratory function monitoring that is useful for intensive care units and for monitoring at homes, and which aid in the detection of sudden deterioration of physiological conditions due to changes in respiration rates. The teachings hereof provide an effective tool for non-contact respiratory function study and analysis.

Claims

1. A method for estimating respiration rate of a subject of interest being monitored for respiratory function in a non-contact, remote sensing environment, the method comprising:

receiving a video of a target region of a body of a subject of interest being monitored for respiratory function, wherein said video is captured using a video camera and an illuminator configured to project a pattern of structured illumination, said video camera being sensitive to electromagnetic radiation in a wavelength of said structured illumination, each image of said captured video comprising a sampling of radiation emitted by a reflection of said structured illumination off a surface of said target region, a spatial distortion being introduced by a reflection of said projected pattern off said surface;

processing said video images to reconstruct 3D depth information of the scene and estimate 3D time-series data from said 3D depth information for said target region; and

estimating said subject'"'"'s respiration rate from said 3D time-series data comprising;

for each image of said video;

comparing spatial attributes of said spatial distortion to known spatial attributes of undistorted projected patterns such that said distortion can be characterized in said image;

calculating a depth map from said characterized distortion at different locations on said surface of said target region;

estimating a 3D volume from said depth map; and

concatenating said resulting estimated 3D volumes to obtain said estimated 3D time-series data.

2. The method of claim 1, wherein said video camera comprises one of:

a video camera that is sensitive in a visible wavelength range, and an IR video camera that is sensitive in an infrared wavelength range.

3. The method of claim 2, wherein said infrared wavelength range comprises any of:

near infrared (NIR), short range infrared red (SWIR), mid-range infrared (MWIR), and long wave infrared (LWIR).

4. The method of claim 1, wherein said target region comprises one of:

said subject'"'"'s anterior thoracic region, a region of said subject'"'"'s dorsal body, and a side view containing said subject'"'"'s thoracic region.

5. The method of claim 4, further comprising:

de-trending said obtained 3D time-series data to remove low frequency variations from said data;

performing a Fast Fourier Transform on said de-trended time-series data; and

performing automatic peak detection to extract said respiration rate.

6. The method of claim 1, further comprising monitoring incremental changes in said respiration rate for an occurrence of any of:

PUHD Type I and PUHD Type II.

7. The method of claim 1, wherein said video is captured using multiple video cameras with non-zero disparity among them with respect to the scene to form stereo vision of the scene, said video cameras being sensitive to the same electromagnetic radiation in a wavelength of interest.

8. The method of claim 1, further comprising monitoring said respiration signal for detecting sudden infant death.

9. The method of claim 1, further comprising monitoring said respiration signal for detecting respiratory function in sleep apnea patients.

10. A system for estimating respiration rate of a subject of interest being monitored for respiratory function in a non-contact, remote sensing environment, the system comprising:

an illuminator configured to project a pattern of structured illumination;

a video camera for capturing a video of a target region of a subject of interest being monitored for respiratory function, said video camera being sensitive to at least the electromagnetic radiation in a wavelength of said structured illumination, wherein said video is captured using a video camera and an illuminator configured to project a pattern of structured illumination, said video camera being sensitive to electromagnetic radiation in a wavelength of said structured illumination; and

a processor in communication with said video camera and a display device, said processor executing machine readable program instructions for performing;

receiving video captured by said video camera, each image of said video comprising a sampling of radiation emitted by a reflection of said structured illumination off a surface of said target region, a spatial distortion being introduced by a reflection of said projected pattern off said surface;

processing said video to estimate 3D time-series data for said target region;

estimating a respiration rate from said 3D time-series data comprising;

for each image of said video;

comparing spatial attributes of said spatial distortion to known spatial attributes of undistorted projected patterns such that said distortion can be characterized in said image;

calculating a depth map from said characterized distortion at different locations on said surface of said target region;

estimating a 3D volume from said depth map; and

concatenating said resulting estimated 3D volumes to obtain said estimated 3D time-series data; and

communicating said estimated respiration rate to said display device.

11. The system of claim 10, wherein said video camera comprises one of:

a video camera that is sensitive in a visible wavelength range, and an IR video camera that is sensitive in an infrared wavelength range.

12. The system of claim 11, wherein said infrared wavelength range comprises any of:

near infrared (NIR), short range infrared red (SWIR), mid-range infrared (MWIR), and long wave infrared (LWIR).

13. The system of claim 10, wherein said target region comprises one of:

said subject'"'"'s anterior thoracic region, a region of said subject'"'"'s dorsal body, and a side view containing said subject'"'"'s thoracic region.

14. The system of claim 13, further comprising:

de-trending said obtained 3D time-series data to remove low frequency variations from said data;

performing a Fast Fourier Transform on said de-trended time-series data; and

performing automatic peak detection to extract said respiration rate.

15. The system of claim 10, further comprising monitoring incremental changes in said respiration rate for an occurrence of any of:

PUHD Type I and PUHD Type II.

16. The system of claim 10, wherein said video is captured using multiple video cameras with non-zero disparity among them with respect to the scene to form stereo vision of the scene, said video cameras being sensitive to the same electromagnetic radiation in a wavelength of interest.

17. The system of claim 10, further comprising monitoring said respiration signal for detecting sudden infant death.

18. The system of claim 10, further comprising monitoring said respiration signal for detecting respiratory function in sleep apnea patients.

19. A computer implemented method for estimating respiration rate of a subject of interest being monitored for respiratory function in a non-contact, remote sensing environment, the method comprising:

receiving a video of a target region of a body of a subject of interest being monitored for respiratory function, said video being captured using a video camera and an illuminator configured to project a pattern of structured illumination, said video camera being sensitive to electromagnetic radiation in a wavelength of said structured illumination, each image of said captured video comprising a sampling of radiation emitted by a reflection of said structured illumination off a surface of said target region, a spatial distortion being introduced by a reflection of said projected pattern off said surface;

processing said video images to estimate 3D time-series data for said target region;

estimating said subject'"'"'s respiration rate from said 3D time-series data comprising;

for each image of said video;

comparing spatial attributes of said spatial distortion to known spatial attributes of undistorted projected patterns such that said distortion can be characterized in said image;

calculating a depth map from said characterized distortion at different locations on said surface of said target region;

estimating a 3D volume from said depth map; and

concatenating said resulting estimated 3D volumes to obtain said estimated 3D time-series data; and

communicating any of;

said respiration rate and said 3D time-series data, to a display device for visual review.

20. The computer implemented method of claim 19, further comprising:

de-trending said obtained 3D time-series data to remove low frequency variations from said data;

performing a Fast Fourier Transform on said de-trended time-series data; and

performing automatic peak detection to extract said respiration rate.