Video-based estimation of heart rate variability

• US 8,977,347 B2
• Filed: 06/25/2012
• Issued: 03/10/2015
• Est. Priority Date: 06/25/2012
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Mestha, Lalit Keshav, Kyal, Survi, Xu, Beilei, Madhu, Himanshu J.
• Primary Examiner(s): Le, Long V
• Assistant Examiner(s): PONTIUS, KEVIN LANE
• Application Number:
  US13/532,057
• Publication Number:
  US 20130345568A1
• Time in Patent Office: 988 Days
• Field of Search: None
• US Class Current: 600/473
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0004 characterised by the type of physiological signal transmitted
• CPC Class Codes: A61B 5/02405 Determining heart rate variability
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/7235 Details of waveform analysis A61B5/349 takes precedence
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

What is disclosed is a video-based system and method for estimating heart rate variability from time-series signals generated from video images captured of a subject of interest being monitored for cardiac function. In a manner more fully disclosed herein, low frequency and high frequency components are extracted from a time-series signal obtained by processing a video of the subject being monitored. A ratio of the low and high frequency of the integrated power spectrum within these components is computed. Analysis of the dynamics of this ratio over time is used to estimate heart rate variability. The teachings hereof can be used in a continuous monitoring mode with a relatively high degree of measurement accuracy and find their uses in a variety of diverse applications such as, for instance, emergency rooms, cardiac intensive care units, neonatal intensive care units, and various telemedicine applications.

Claims

1. A method for estimating heart rate variability from a video captured of a subject of interest, the method comprising:

receiving a video captured of a target area of a region of exposed skin of a subject of interest where a photoplethysmographic (PPG) signal of said subject can be registered, said video comprising video images captured by a video camera;

selecting a first batch of said video images for processing;

identifying a first region of interest (ROI) in said selected batch of video images for processing; and

A) processing pixels in said ROI of said selected batch of said video images to obtain a time-series signal containing a PPG signal captured by said imaging channel of said video camera;

B) processing said time-series signal to extract said PPG signal;

C) calculating a power spectral density function on said PPG signal;

D) using a peak detection algorithm to detect low frequency and high frequency components over a pre-defined frequency range from said power spectral density function;

E) in response to a low frequency peak or a high frequency peak not being detected in said power spectral density function, performing at least one of;

(i) identifying a next ROI which is different than said first ROI and repeating again at step (A);

(ii) selecting a next batch of video images which is at least partially different than said first batch of video images and repeating again at step (A)

(iii) using signal reconstruction to introduce one or more missing peaks on said PPG signal and then repeating steps (C), (D) and (F); and

F) determining a ratio of said low and high frequency components over said interval of said selected batch, said ratio comprising said subject'"'"'s estimated heart rate variability for said interval.

2. The method of claim 1, wherein said target area contains at least a partial view of any of:

an anterior thoracic region of said subject, a back region of said subject'"'"'s dorsal body, a side view of a thoracic region of said subject, an exposed region of said subject'"'"'s forearm, and exposed region of said subject'"'"'s palm, a region of said subject'"'"'s face, a region of said subject'"'"'s neck, and a region near a finger of said subject.

3. The method of claim 1, wherein video was captured using a video camera comprising any of:

an infrared video camera with at least one imaging channel capturing NIR images, a color video camera with at least one imaging channel capturing RGB images, a hybrid infrared and color video camera with imaging channels capturing RGB and NIR images, a multispectral video camera with at least one imaging channel capturing multispectral images, and hyperspectral video camera with at least one imaging channel capturing hyperspectral video images.

4. The method of claim 1, wherein said PPG signal is one of:

reflective and transmissive.

5. The method of claim 1, wherein, in advance of extracting said time-series signal, compensating for any of:

a motion induced blur, an imaging blur, and slow illuminant variation.

6. The method of claim 1, further comprising communicating said estimated heart rate variability to a display device.

7. The method of claim 1, wherein, in advance of using signal reconstruction to introduce one or more missing peaks, further comprising de-trending said PPG signal followed by band pass filtering.

8. A system for estimating heart rate variability from a video captured of a subject of interest, the system comprising:

a video camera with at least one imaging channel; and

a processor in communication with a memory, said processor executing machine readable instructions for performing;

receiving a video captured of a target area of a region of exposed skin of a subject of interest where a photoplethysmographic (PPG) signal of said subject can be registered, said video comprising video images captured by said video camera;

selecting a first batch of said video images for processing;

identifying a first region of interest (ROI) in said selected batch of video images for processing; and

A) processing pixels in said ROI of said selected batch of said video images to obtain a time-series signal containing a PPG signal captured by said imaging channel of said video camera;

B) processing said time-series signal to extract said PPG signal;

C) calculating a power spectral density function on said PPG signal;

D) using a peak detection algorithm to detect low frequency and high frequency components over a pre-defined frequency range from said power spectral density function;

E) in response to a low frequency peak or a high frequency peak not being detected in said power spectral density function, performing at least one of;

(i) identifying a next ROI which is different than said first ROI and repeating again at step (A);

(ii) selecting a next batch of video images which is at least partially different than said first batch of video images and repeating again at step (A)

(iii) using signal reconstruction to introduce one or more missing peaks on said PPG signal and then repeating steps (C), (D) and (F); and

F) determining a ratio of said low and high frequency components over said interval of said selected batch, said ratio comprising said subject'"'"'s estimated heart rate variability for said interval.

9. The system of claim 8, wherein said target area contains at least a partial view of any of:

an anterior thoracic region of said subject, a back region of said subject'"'"'s dorsal body, a side view of a thoracic region of said subject, an exposed region of said subject'"'"'s forearm, and exposed region of said subject'"'"'s palm, a region of said subject'"'"'s face, a region of said subject'"'"'s neck, and a region near a finger of said subject.

10. The system of claim 8, wherein said video camera comprises any of:

an infrared video camera with at least one imaging channel capturing NIR images, a color video camera with at least one imaging channel capturing RGB images, a hybrid infrared and color video camera with imaging channels capturing RGB and NIR images, a multispectral video camera with at least one imaging channel capturing multispectral images, and hyperspectral video camera with at least one imaging channel capturing hyperspectral video images.

11. The system of claim 8, wherein said PPG signal is one of:

reflective and transmissive.

12. The system of claim 8, wherein, in advance of extracting said time-series signal, compensating for any of:

a motion induced blur, an imaging blur, and slow illuminant variation.

13. The system of claim 8, further comprising communicating said estimated heart rate variability to a display device.

14. The system of claim 8, wherein, in advance of using signal reconstruction to introduce one or more missing peaks, further comprising de-trending said PPG signal followed by band pass filtering.

15. A computer implemented method for estimating heart rate variability from a video captured of a subject of interest, the method comprising:

receiving a video captured of a target area of a region of exposed skin of a subject of interest where a photoplethysmographic (PPG) signal of said subject can be registered, said video comprising video images captured by a video camera;

selecting a first batch of said video images for processing;

identifying a first region of interest (ROI) in said selected batch of video images for processing; and

A) processing pixels in said ROI of said selected batch of said video images to obtain a time-series signal containing a PPG signal captured by said imaging channel of said video camera;

B) processing said time-series signal to extract said PPG signal;

C) calculating a power spectral density function on said PPG signal;

D) using a peak detection algorithm to detect low frequency and high frequency components over a pre-defined frequency range from said power spectral density function;

E) in response to a low frequency peak or a high frequency peak not being detected in said power spectral density function, performing at least one of;

(i) identifying a next ROI which is different than said first ROI and repeating again at step (A);

(ii) selecting a next batch of video images which is at least partially different than said first batch of video images and repeating again at step (A)

(iii) using signal reconstruction to introduce one or more missing peaks on said PPG signal and then repeating steps (C), (D) and (F); and

F) determining a ratio of said low and high frequency components over said interval of said selected batch, said ratio comprising said subject'"'"'s estimated heart rate variability for said interval.

16. The computer implemented method of claim 15, wherein said target area contains at least a partial view of any of:

an anterior thoracic region of said subject, a back region of said subject'"'"'s dorsal body, a side view of a thoracic region of said subject, an exposed region of said subject'"'"'s forearm, and exposed region of said subject'"'"'s palm, a region of said subject'"'"'s face, a region of said subject'"'"'s neck, and a region near a finger of said subject.

17. The computer implemented method of claim 15, wherein video was captured using a video camera comprising any of:

an infrared video camera with at least one imaging channel capturing NIR images, a color video camera with at least one imaging channel capturing RGB images, a hybrid infrared and color video camera with imaging channels capturing RGB and NIR images, a multispectral video camera with at least one imaging channel capturing multispectral images, and hyperspectral video camera with at least one imaging channel capturing hyperspectral video images.

18. The computer implemented method of claim 15, wherein said PPG signal is one of:

reflective and transmissive.

19. The computer implemented method of claim 15, wherein, in advance of extracting said time-series signal, compensating for any of:

a motion induced blur, an imaging blur, and slow illuminant variation.

20. The computer implemented method of claim 15, wherein, in advance of using signal reconstruction to introduce one or more missing peaks, further comprising de-trending said PPG signal followed by band pass filtering.