System and methods for vital sign estimation from passive thermal video

• US 8,149,273 B2
• Filed: 07/10/2008
• Issued: 04/03/2012
• Est. Priority Date: 11/30/2007
• Status: Expired due to Fees

• Current Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Liu, Qiong, Yang, Ming, Turner, Althea
• Primary Examiner(s): LIN, WEN TAI
• Application Number:
  US12/171,273
• Publication Number:
  US 20090141124A1
• Time in Patent Office: 1,363 Days
• Field of Search: None
• US Class Current: 348/65
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/489 Blood vessels

Abstract

A system for measuring a pulse and respiratory rate from passive thermal video includes contour segmentation and tracking, clustering of informative pixels of interests, and robust dominant frequency component estimation. Contour segmentation is used to locate a blood vessel region to measure, after which all pixels in the nearby region are aligned across frames based on the segmentation'"'"'s position, and scale in each frame. Spatial filtering is then performed to remove noise not related to heart beat and then non-linear filtering is performed on the temporal signal corresponding to each aligned pixel. The signal spectrum of each pixel is then feed to a clustering algorithm for outlier removal. Pixels in the largest cluster are then used to vote for the dominant frequency, and the median of the dominant frequency is output as the pulse rate.

Claims

1. A system for estimating vital signs from passive thermal video comprising:

an infrared video receiving module for receiving an infrared video segment of a subject;

a contour segmentation module for contour segmenting the video segment to select a region of pixels representing a portion of the subject;

an alignment module for aligning the selected region of pixels across a plurality of frames of the video segment;

a signal detection module for detecting a signal from the selected region using a thermal wave propagation-based signal detection method;

a spatial filtering module for spatial filtering the signal to remove noise not related to the vital signs to be estimated;

a non-linear temporal filtering module to process a temporal signal corresponding to each aligned pixel sequence;

an clustering module for removing outliers of the signal using a pixel clustering algorithm;

a frequency selection module for selecting a frequency peak of the signal using a dominant frequency voting technique that generates a plurality of bins based on a sampling rate, each of the plurality of bins representing a possible frequency peak of the signal, votes on each of the plurality of bins and selects the frequency peak having a highest number of votes; and

an averaging module for averaging the selected frequency peak to compute an average estimation of at least one vital sign.

2. The system of claim 1, wherein the contour segmentation module further includes a natural feature module for selecting natural features close to the selected region.

3. The system of claim 2, wherein the contour segmentation module further includes a temperature selection module for selecting an isotherm temperature based on temperature modulation dynamic range and camera sensitivity.

4. The system of claim 3, wherein the contour segmentation module further includes a contour selection module for selecting a contour line corresponding to the steepest temperature change in the selected region.

5. The system of claim 4, wherein the selected region of pixels corresponds to a blood vessel.

6. The system of claim 1, wherein the spatial filtering module further includes a frequency response computation module for computing the frequency responses of the signal using a normalized rectangular window and a Hamming window.

19. The system of claim 4, wherein the selected region of pixels corresponds to a nostril.

20. The system of claim 1, wherein averaging the selected frequency peak comprises averaging the selected frequency peak with adjacent ones of the possible frequency peaks in numerical value.

7. A method for estimating vital signs from a passive thermal video comprising the steps of

receiving an infrared video segment of a subject;

contour segmenting the video segment to select a region of pixels representing a portion of the subject;

aligning the selected region of pixels across a plurality of frames of the video segment;

detecting a signal from the selected region using a thermal wave propagation-based signal detection method;

spatial filtering the signal to remove noise not related to the vital signs to be estimated;

performing non-linear filtering on the signal corresponding to each aligned pixel of the selected region;

removing outliers of the signal using a pixel clustering algorithm;

selecting a frequency peak of the signal using a dominant frequency voting technique that generates a plurality of bins based on a sampling rate, each of the plurality of bins representing a possible frequency peak of the signal, votes on each of the plurality of bins and selects the frequency peak having a highest number of votes; and

averaging the selected frequency peak to compute an average estimation of at least one vital sign.

8. The method of claim 7, wherein the step of contour segmenting the video segment further includes selecting natural features close to the selected region.

9. The method of claim 8, wherein the step of contour segmenting the video segment further includes selecting an isotherm temperature based on temperature modulation dynamic range and camera sensitivity.

10. The method of claim 9, wherein the step of contour segmenting the video segment further includes selecting a contour line corresponding to the steepest temperature change in the selected region.

11. The method of claim 10, wherein the step of contour segmenting the video segment further includes selecting a region of pixels corresponding to a blood vessel.

12. The method of claim 7, wherein the step of spatial filtering the signal to remove noise further includes computing the frequency responses of the signal using a normalized rectangular window and a Hamming window.

13. A computer program product embodied on a non-transitory computer readable medium for estimating vital signs from passive thermal video comprising computer program code for:

receiving an infrared video segment of a subject;

contour segmenting the video segment to select a region of pixels representing a portion of the subject;

aligning the selected region of pixels across a plurality of frames of the video segment;

detecting a signal from the selected region using a thermal wave propagation-based signal detection method;

spatial filtering the signal to remove noise not related to the vital signs to be estimated;

performing non-linear filtering on the signal corresponding to each aligned pixel of the selected region;

removing outliers of the signal using a pixel clustering algorithm;

selecting a frequency peak of the signal using a dominant frequency voting technique that generates a plurality of bins based on a sampling rate, each of the plurality of bins representing a possible frequency peak of the signal, votes on each of the plurality of bins and selects the frequency peak having a highest number of votes; and

averaging the selected frequency peak to compute an average estimation of at least one vital sign.

14. The computer program product of claim 13, further comprising computer program code for contour segmenting the video segment by selecting natural features close to the selected region.

15. The computer program product of claim 14, further comprising computer program code for contour segmenting the video segment by selecting an isotherm temperature based on temperature modulation dynamic range and camera sensitivity.

16. The computer program product of claim 15, further comprising computer program code for contour segmenting the video segment by selecting a contour line corresponding to the steepest temperature change in the selected region.

17. The computer program product of claim 16, further comprising computer program code for selecting a region of pixels that corresponds to a blood vessel.

18. The computer program product of claim 13, further comprising computer program code for spatial filtering the signal to remove noise by computing the frequency responses of the signal using a normalized rectangular window and a Hamming window.