VIDEO-BASED ESTIMATION OF HEART RATE VARIABILITY

• US 20130345568A1
• Filed: 06/25/2012
• Published: 12/26/2013
• Est. Priority Date: 06/25/2012
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: MESTHA, Lalit Keshav, KYAL, Survi, XU, Beilei, MADHU, Himanshu J.
• Granted Patent: US 8,977,347 B2
• Time in Patent Office: Days
• US Class Current: 600/473
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0004 characterised by the type of physiological signal transmitted
• CPC Class Codes: A61B 5/02405 Determining heart rate variability
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/7235 Details of waveform analysis A61B5/349 takes precedence
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

What is disclosed is a video-based system and method for estimating heart rate variability from time-series signals generated from video images captured of a subject of interest being monitored for cardiac function. In a manner more fully disclosed herein, low frequency and high frequency components are extracted from a time-series signal obtained by processing a video of the subject being monitored. A ratio of the low and high frequency of the integrated power spectrum within these components is computed. Analysis of the dynamics of this ratio over time is used to estimate heart rate variability. The teachings hereof can be used in a continuous monitoring mode with a relatively high degree of measurement accuracy and find their uses in a variety of diverse applications such as, for instance, emergency rooms, cardiac intensive care units, neonatal intensive care units, and various telemedicine applications.

Claims

1. A method for estimating heart rate variability from a video captured of a subject of interest being monitored in a non-contact remote sensing environment, the method comprising:

receiving a video captured of a target area of a region of exposed skin of a subject of interest where a photoplethysmographic (PPG) signal of said subject can be registered, said video comprising video images captured by a video camera with at least one imaging channel;

processing said video images to generate a time-series signal containing a PPG signal captured by said imaging channel;

processing said PPG signal to obtain a nearly stationary PPG signal comprising;

P_stat=(I−

(I+λ

²D₂^TD₂)<sup>−

1</sup>P_original

where, I is an identity matrix, P_original is a normalized zero-removed time-series signal, λ

is a user-defined parameter used to adjust a frequency response of said de-trending algorithm, and D₂ is a second order difference matrix;

extracting, over a pre-defined time interval, low frequency and high frequency components of an integrated power spectrum within said time-series signal; and

determining a ratio of said low and high frequency components over said interval, said ratio comprising said subject'"'"'s estimated heart rate variability for said interval.

2. The method of claim 1, wherein said target area contains at least a partial view of any of:

an anterior thoracic region of said subject, a back region of said subject'"'"'s dorsal body, a side view of a thoracic region of said subject, an exposed region of said subject'"'"'s forearm, and exposed region of said subject'"'"'s palm, a region of said subject'"'"'s face, a region of said subject'"'"'s neck, and a region near a finger of said subject.

3. The method of claim 1, wherein video was captured using a video camera comprising any of:

an infrared video camera with at least one imaging channel capturing NIR images, a color video camera with at least one imaging channel capturing RGB images, a hybrid infrared and color video camera with imaging channels capturing RGB and NIR images, a multispectral video camera with at least one imaging channel capturing multispectral images, and hyperspectral video camera with at least one imaging channel capturing hyperspectral video images.

4. The method of claim 1, wherein said PPG signals are one of:

reflective and transmissive.

5. The method of claim 1, wherein, in advance of extracting said time-series signal, compensating for any of:

a motion induced blur, an imaging blur, and slow illuminant variation.

8. The method of claim 1, further comprising communicating said estimated heart rate variability to a display device.

6. (canceled)

7. The method of claim 6, wherein said second order difference matrix comprises:

9. A system for estimating heart rate variability from a video captured of a subject of interest being monitored in a non-contact remote sensing environment, the system comprising:

a video camera with at least one imaging channel; and

a processor in communication with a memory, said processor executing machine readable instructions for performing;

receiving a video captured of a target area of a region of exposed skin of a subject of interest where a photoplethysmographic (PPG) signal of said subject can be registered, said video comprising video images captured by said video camera;

processing said video images to generate a time-series signal containing a PPG signal captured by said imaging channel;

processing said PPG signal to obtain a nearly stationary PPG signal comprising;

P_stat=(I−

(I+λ

²D₂^TD₂)<sup>−

P</sup>_original

where, I is an identity matrix, P_original is a normalized zero-removed time-series signal, λ

is a user-defined parameter used to adjust a frequency response of said de-trending algorithm, and D₂ is a second order difference matrix;

extracting, over a pre-defined time interval, low frequency and high frequency components of an integrated power spectrum within said time-series signal; and

determining a ratio of said low and high frequency components over said interval, said ratio comprising said subject'"'"'s estimated heart rate variability for said interval.

10. The system of claim 9, wherein said target area contains at least a partial view of any of:

an anterior thoracic region of said subject, a back region of said subject'"'"'s dorsal body, a side view of a thoracic region of said subject, an exposed region of said subject'"'"'s forearm, and exposed region of said subject'"'"'s palm, a region of said subject'"'"'s face, a region of said subject'"'"'s neck, and a region near a finger of said subject.

11. The system of claim 9, wherein said video camera comprises any of:

an infrared video camera with at least one imaging channel capturing NIR images, a color video camera with at least one imaging channel capturing RGB images, a hybrid infrared and color video camera with imaging channels capturing RGB and NIR images, a multispectral video camera with at least one imaging channel capturing multispectral images, and hyperspectral video camera with at least one imaging channel capturing hyperspectral video images.

12. The system of claim 9, wherein said PPG signals are one of:

reflective and transmissive.

13. The system of claim 9, wherein, in advance of extracting said time-series signal, compensating for any of:

a motion induced blur, an imaging blur, and slow illuminant variation.

16. The system of claim 9, further comprising communicating said estimated heart rate variability to a display device.

14. (canceled)

15. The system of claim 14, wherein said second order difference matrix comprises:

17. A computer implemented method for estimating heart rate variability from a video captured of a subject of interest being monitored in a non-contact remote sensing environment, the method comprising:

receiving a video captured of a target area of a region of exposed skin of a subject of interest where a photoplethysmographic (PPG) signal of said subject can be registered, said video comprising video images captured by a video camera with at least one imaging channel;

processing said video images to generate a time-series signal containing PPG signal captured by said imaging channel;

de-trending said PPG signal to remove slow non-stationary trends from said signal to obtain a nearly stationary PPG signal comprising;

P_stat=(I−

(I+λ

²D₂^TD₂)<sup>−

1</sup>P_original

where, I is an identity matrix, P_original is a normalized zero-removed time-series signal, λ

is a user-defined parameter used to adjust a frequency response of said de-trending algorithm, and D₂ is a second order difference matrix;

extracting, over a pre-defined time interval, low frequency and high frequency components of the integrated power spectrum within said time-series signal;

determining a ratio of said low and high frequency components over said interval, said ratio comprising said subject'"'"'s estimated heart rate variability for said interval; and

communicating said estimated heart rate variability to a display device.

18. The computer implemented method of claim 17, wherein said target area contains at least a partial view of any of:

an anterior thoracic region of said subject, a back region of said subject'"'"'s dorsal body, a side view of a thoracic region of said subject, an exposed region of said subject'"'"'s forearm, and exposed region of said subject'"'"'s palm, a region of said subject'"'"'s face, a region of said subject'"'"'s neck, and a region near a finger of said subject.

19. The computer implemented method of claim 17, wherein video was captured using a video camera comprising any of:

an infrared video camera with at least one imaging channel capturing NIR images, a color video camera with at least one imaging channel capturing RGB images, a hybrid infrared and color video camera with imaging channels capturing RGB and NIR images, a multispectral video camera with at least one imaging channel capturing multispectral images, and hyperspectral video camera with at least one imaging channel capturing hyperspectral video images.

20. The computer implemented method of claim 17, wherein said PPG signals are one of:

reflective and transmissive.

21. The computer implemented method of claim 17, wherein, in advance of extracting said time-series signal, compensating for any of:

a motion induced blur, an imaging blur, and slow illuminant variation.

22. (canceled)

23. The computer implemented method of claim 22, wherein said second order difference matrix comprises: