Processing a video for tidal chest volume estimation

• US 9,226,691 B2
• Filed: 06/01/2012
• Issued: 01/05/2016
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Bernal, Edgar A., Mestha, Lalit Keshav, Xu, Beilei
• Primary Examiner(s): Brutus, Joel F
• Application Number:
  US13/486,637
• Publication Number:
  US 20130324876A1
• Time in Patent Office: 1,313 Days
• Field of Search: 600/411, 600/376, 600/427, 600/407
• US Class Current: 1/1
• CPC Class Codes: A61B 5/0064 Body surface scanning
• CPC Class Codes: A61B 5/091 Measuring volume of inspired or expired gases, e.g. to determine lung capacity
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/1135 by monitoring thoracic expansion
• CPC Class Codes: G01B 11/2527 with phase change by in-plane movement of the patern

Abstract

What is disclosed is a system and method for estimating tidal chest volume using 3D surface reconstruction based on an analysis of captured reflections of structured illumination patterns from the subject with a video camera. The imaging system hereof captures the reflection of the light patterns from a target area of the subject'"'"'s thoracic region. The captured information produces a depth map and a volume is estimated from the resulting 3D map. The teachings hereof provide a non-contact approach to patient respiration monitoring that is particularly useful for infant care in a neo-natal intensive care unit (NICU), and can aid in the early detection of sudden deterioration of physiological condition due to detectable changes in respiratory function. The systems and methods disclosed herein provide an effective tool for tidal chest volume study and respiratory function analysis.

Claims

1. A method for determining chest volume from video captured of a subject of interest being monitored for respiratory function, the method comprising:

manipulating, in order to reduce image artifacts, at least one structured illumination source and at least one unstructured illumination source illuminating a scene being captured by a video system actively acquiring a video of a target region containing at least a partial view of a thoracic region of a subject of interest being monitored for respiratory function, captured images corresponding to at least an inspiration and expiration cycle of said subject, wherein said manipulating includes spectrally multiplexing said at least one structured illumination source and at least one unstructured illumination source by making a wavelength range of said at least one structured illumination source narrower than a wavelength range of said at least one unstructured illumination source such that reflected energy from both illumination sources can be recovered via filtering, said at least one structured illumination source and at least one unstructured illumination source having significantly overlapping spectral bands;

for each of said captured images, comparing spatial characteristics of said reflection in said captured images to known spatial characteristics of undistorted projected patterns such that a spatial distortion of captured patterns can be characterized, said spatial distortion having been introduced by a reflection of said captured patterns off a surface of said target region;

calculating a sequence of depth maps from said spatial distortion, in depth map being characterized for each of said captured images;

for each depth map in said sequence of depth maps;

determining spatial coordinates corresponding to said depth map; and

estimating a 3D chest volume for each depth map in said sequence of depth maps;

calculating said subject'"'"'s chest volume using a proportionality constant which converts estimated 3D volumes to an approximation of a measured chest volume as measured by laboratory tests; and

storing, using a processor, said subject'"'"'s chest volume to a storage device.

2. The method of claim 1, wherein said target region comprises any of:

an anterior thoracic region of said subject, a back region of said subject'"'"'s dorsal body, and a side view of said thoracic region.

3. The method of claim 1, wherein estimating said 3D chest volume for each of said depth maps comprises:

tessellating surface points of said depth map based on said determined spatial coordinates;

computing a reference point location based on said determined spatial coordinates;

connecting triangles in tessellation with said reference point to form tetrahedrons;

computing a volume of each of said tetrahedrons; and

aggregating tetrahedral volumes to obtain said estimated 3D chest volume for said depth map.

4. The method of claim 3, wherein said reference point location comprises one of:

a centroid, a weighted arithmetic mean, and a rank-ordered statistic of surface point locations.

5. The method of claim 1, further comprising, in advance of estimating 3D volume, segmenting said depth map into regions corresponding to said subject'"'"'s target region.

6. The method of claim 1, further comprising tracking a tidal chest volume over time to estimate any of:

a respiration cycle and a respiration rate of said subject.

7. The method of claim 1, further comprising compensating for an effect of a body motion of said subject by any of:

video-based image stabilization, and 3D surface stabilization.

8. The method of claim 1, wherein estimating said proportionality constant comprises:

measuring said subject'"'"'s lung volumes by a laboratory test over various stages of inspiration and expiration;

performing linear regression between said measured lung volumes and said estimated 3D chest volumes; and

calculating a slope of said linear regression to obtain said proportionality constant.

9. A system to determine chest volume from video captured of a subject of interest being monitored for respiratory function, the system comprising:

a video camera and an illuminator configured to project a pattern of structured illumination, said video camera being, at least in part, sensitive to electromagnetic radiation in a wavelength range contains the wavelength of said structured illumination to capture time-varying video of a subject of interest being monitored for respiratory function; and

a processor in communication with a memory, said processor executing machine readable instructions to perform;

receive a video captured by said video camera of a target region containing at least a partial view of a thoracic region of said subject, each captured image comprising sampled radiation emitted by a reflection of a patterned structured illumination off said target region, said captured images corresponding to at least an inspiration and expiration cycle of said subject;

manipulate, in order to reduce image artifacts, at least one structured illumination source and at least one unstructured illumination source illuminating a scene being captured by a video system actively acquiring a video of a target region containing at least a partial view of a thoracic region of a subject, said captured images corresponding to at least an inspiration and expiration cycle of said subject, wherein said manipulating includes spectrally multiplexing said at least one structured illumination source and at least one unstructured illumination source by making a wavelength range of said at least one structured illumination source narrower than a wavelength range of said at least one unstructured illumination source such that reflected energy from both illumination sources can be recovered via filtering, said at least one structured illumination source and at least one unstructured illumination source having significantly overlapping spectral bands;

compare spatial characteristics of said reflection in said captured images to known spatial characteristics of undistorted projected patterns such that a spatial distortion of captured patterns can be characterized, said spatial distortion having been introduced by a reflection of said captured patterns off a surface of said target region;

calculate a sequence of depth maps from said spatial distortion, in depth map being characterized for each of said captured images;

for each depth map in said sequence of depth maps;

determine spatial coordinates corresponding to said depth map; and

estimate a 3D chest volume for each depth map in said sequence of depth maps;

calculate said subject'"'"'s chest volume using a proportionality constant which converts estimated 3D volumes to an approximation of a measured chest volume as measured by laboratory tests; and

store said subject'"'"'s chest volume to a storage device.

10. The system of claim 9, wherein said target region comprises any of:

an anterior thoracic region of said subject, a back region of said subject'"'"'s dorsal body, and a side view of said thoracic region.

11. The system of claim 9, wherein estimating said 3D chest volume for each of said depth maps comprises:

tessellating surface points of said depth map based on said determined spatial coordinates;

computing a reference point location based on said determined spatial coordinates;

connecting triangles in tessellation with said reference point to form tetrahedrons;

computing a volume of each of said tetrahedrons; and

aggregating tetrahedral volumes to obtain said estimated 3D chest volume for said depth map.

12. The system of claim 11, wherein said reference point location comprises one of:

a centroid, a weighted arithmetic mean, and a rank-ordered statistic of surface point locations.

13. The system of claim 9, further comprising, in advance of estimating said 3D volume, segmenting said depth map into regions corresponding to said subject'"'"'s target region.

14. The system of claim 9, further comprising tracking a tidal chest volume over time to estimate any of:

a respiration cycle and a respiration rate of said subject.

15. The system of claim 9, further comprising compensating for an effect of a body motion of said subject by any of:

video image stabilization, and 3D surface stabilization.

16. A computer implemented method for determining chest volume from video captured of a subject of interest being monitored for respiratory function, the method comprising:

manipulating, in order to reduce image artifacts, at least one structured illumination source and at least one unstructured illumination source illuminating a scene being captured by a video system actively acquiring a video of a target region containing at least a partial view of a thoracic region of a subject of interest being monitored for respiratory function, captured images corresponding to at least an inspiration and expiration cycle of said subject, wherein said manipulating includes spectrally multiplexing said illumination sources said at least one structured illumination source and at least one unstructured illumination source by making a wavelength range of said at least one structured illumination source narrower than a wavelength range of said at least one unstructured illumination source such that reflected energy from both illumination sources can be recovered via filtering, said at least one structured illumination source and at least one unstructured illumination source having significantly overlapping spectral bands;

for each of said captured images, comparing spatial characteristics of said reflection in said captured images to known spatial characteristics of undistorted projected patterns such that a spatial distortion of captured patterns can be characterized, said spatial distortion having been introduced by a reflection of said captured patterns off a surface of said target region;

calculating a sequence of depth maps from said spatial distortion, in depth map being characterized for each of said captured images;

for each depth map in said sequence of depth maps;

determining spatial coordinates corresponding to said depth map; and

estimating a 3D chest volume for each depth map in said sequence of depth maps; and

calculating said subject'"'"'s chest volume using a proportionality constant which converts said estimated 3D volumes to an approximation of a measured chest volume as measured by laboratory tests.

17. The computer implemented method of claim 16, wherein estimating said 3D chest volume for each of said depth maps comprises:

tessellating surface points of said depth map based on said determined spatial coordinates;

computing a reference point location based on said determined spatial coordinates;

connecting triangles in tessellation with said reference point to form tetrahedrons;

computing a volume of each of said tetrahedrons; and

aggregating tetrahedral volumes to obtain said estimated 3D chest volume for said depth map.

18. The computer implemented method of claim 16, further comprising tracking a tidal chest volume over time to estimate any of:

a respiration cycle and a respiration rate of said subject.

19. The computer implemented method of claim 16, further comprising compensating for an effect of a body motion of said subject by any of:

video image stabilization, and 3D surface stabilization.