REMOVING ENVIRONMENT FACTORS FROM SIGNALS GENERATED FROM VIDEO IMAGES CAPTURED FOR BIOMEDICAL MEASUREMENTS

• US 20130215244A1
• Filed: 02/21/2012
• Published: 08/22/2013
• Est. Priority Date: 02/21/2012
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: MESTHA, Lalit Keshav, Xu, Beilei
• Granted Patent: US 9,185,353 B2
• Time in Patent Office: Days
• US Class Current: 348/77
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

What is disclosed is a system and method for automatically removing undesirable periodic or random background noise from heart rate measurement signals obtained from a video camera, ambient illuminator and other unknown electromagnetic sources to improve the overall reliability of biomedical measurements. In one embodiment, a time varying video image acquired over at least one imaging channel of a subject of interest is received. The video images are then segmented into a first region comprising a localized area where plethysmographic signals of the subject can be registered and a second region comprising a localized area of the environment where the plethysmographic signals cannot be registered. Both of the regions are exposed to the same environmental factors. The segmented video signals are pre-processed and the processed signals are subtracted from each other to generate an environmentally compensated signal. The environmentally compensated signal is then communicated to a computer system.

Claims

1. A method for removing environmental factors from signals generated from video images captured by a non-contact imaging system in a remote sensing environment, the method comprising:

receiving video images captured of a subject of interest, said video comprising a time varying source video image acquired over at least one imaging channel;

segmenting said video images into a first region of interest comprising a localized area where plethysmographic signals of said subject can be registered and a second region of interest comprising a localized area where said subject'"'"'s plethysmographic signals cannot be registered, both of said first and second regions being exposed to the same environmental factors;

pre-processing said video images to generate a time-series signal for each of said first and second regions;

subtracting said time-series signal corresponding to each of said first and second regions to obtain an environmentally compensated signal; and

communicating said environmentally compensated signal to a computer system.

2. The method of claim 1, wherein pre-processing said video images is performed using at least one of:

a source separation with blind source separation, and a constrained source separation.

3. The method of claim 2, wherein said pre-processing further comprises, for each channel:

computing an average value of all pixels acquired with this channel to obtain a channel average per image frame;

computing a global channel average and a global standard deviation from said computed averages for this channel;

subtracting said channel average from said global channel average to produce a resulting signal;

dividing said resulting signal by said standard deviation to obtain a zero-mean unit variance time-series signal for said region;

normalizing said time-series signal; and

band-pass filtering said normalized time-series signal to remove frequencies that are above and below expected frequencies of the plethysmographic signals of said subject.

4. The method of claim 1, wherein said acquired source video image comprises any combination of:

NIR images, RGB images, RGB and NIR images, multi-spectral images, and hyperspectral video images.

5. The method of claim 1, wherein, in advance of segmenting said video images, further comprising compensating for any of:

a motion induced blur, an imaging blur, and slow illuminant variation.

6. The method of claim 5, wherein said compensation is carried out in a time-domain before performing a Fourier transform such that both a periodic and a non-periodic background signal can be reduced.

7. The method of claim 1, further comprising performing a Fourier Transform on said source signal to remove periodic noise in advance of said subtraction.

8. The method of claim 1, further comprising:

generating a set of reference signals having a frequency range which approximates a frequency range of said subject'"'"'s cardiac pulse;

performing, using said reference signal, a constrained source separation on said source data to obtain an estimated source signal with a minimum error; and

estimating a cardiac frequency of said subject based upon a frequency at which said minimum error was achieved.

9. The method of claim 8, wherein said minimum error was achieved by adjusting phase of the estimated source signal and calculating a difference between two waveforms.

10. The method of claim 8, further comprising changing at least one aspect of said reference signal by changing any of:

a frequency, an amplitude, a phase, and a wave form of said reference signal.

11. The method of claim 10, wherein said wave form comprises any of:

a sine wave, a square wave, a user defined shape such as that obtained from an ECG signal, and a cardiac pulse wave form derived from said subject.

12. A system for removing environmental factors from signals generated from video images captured by a non-contact imaging system in a remote sensing environment, the system comprising:

an imaging sensor for acquiring a time varying source video image acquired over at least one imaging channel; and

a processor in communication with said imaging sensor and a memory, said processor executing machine readable instructions for performing;

receiving video images captured of a subject of interest using said sensor;

segmenting said video images into a first region of interest comprising a localized area where plethysmographic signals of said subject can be registered and a second region of interest comprising a localized area where said subject'"'"'s plethysmographic signals cannot be registered, both of said first and second regions being exposed to the same environmental factors;

pre-processing said video images to generate a time-series signal for each of said first and second regions;

subtracting said time-series signal corresponding to each of said first and second regions to obtain an environmentally compensated signal; and

communicating said environmentally compensated signal to a computer system.

13. The system of claim 12, wherein pre-processing said video images is performed using at least one of:

a source separation with blind source separation, and a constrained source separation.

14. The system of claim 13, wherein said pre-processing further comprises, for each channel:

computing an average value of all pixels acquired with this channel to obtain a channel average per image frame;

computing a global channel average and a global standard deviation from said computed averages for this channel;

subtracting said channel average from said global channel average to produce a resulting signal;

dividing said resulting signal by said standard deviation to obtain a zero-mean unit variance time-series signal for said region;

normalizing said time-series signal; and

band-pass filtering said normalized time-series signal to remove frequencies that are above and below expected frequencies of the plethysmographic signals of said subject.

15. The system of claim 12, wherein said acquired source video image comprises any combination of:

NIR images, RGB images, RGB and NIR images, multi-spectral images, and hyperspectral video images.

16. The system of claim 12, wherein, in advance of segmenting said video images, further comprising compensating for any of:

a motion induced blur, an imaging blur, and slow illuminant variation.

17. The system of claim 16, wherein said compensation is carried out in a time-domain before performing a Fourier transform such that both a periodic and a non-periodic background signal can be reduced.

18. The system of claim 12, further comprising performing a Fourier Transform on said source signal to remove periodic noise in advance of said subtraction.

19. The system of claim 12, further comprising:

generating a set of reference signals having a frequency range which approximates a frequency range of said subject'"'"'s cardiac pulse;

performing, using said reference signal, a constrained source separation on said source data to obtain an estimated source signal with a minimum error; and

estimating a cardiac frequency of said subject based upon a frequency at which said minimum error was achieved.

20. The system of claim 19, wherein said minimum error was achieved by adjusting phase of the estimated source signal and calculating a difference between two waveforms.

21. The system of claim 19, further comprising changing at least one aspect of said reference signal by changing any of:

a frequency, an amplitude, a phase, and a wave form of said reference signal.

22. The system of claim 21, wherein said wave form comprises any of:

a sine wave, a square wave, a user defined shape such as that obtained from an ECG signal, and a cardiac pulse wave form derived from said subject.

23. A computer implemented method for removing environmental factors from signals generated from video images captured by a non-contact imaging system in a remote sensing environment, the method comprising:

receiving video images captured of a subject of interest, said video comprising a time varying source video image acquired over at least one imaging channel, said acquired source video image comprising any combination of;

NIR images, RGB images, RGB and NIR images, multi-spectral images, and hyperspectral video images;

segmenting said video images into a first region of interest comprising a localized area where plethysmographic signals of said subject can be registered and a second region of interest comprising a localized area where said subject'"'"'s plethysmographic signals cannot be registered, both of said first and second regions being exposed to the same environmental factors;

pre-processing said video images to generate a normalized time-series signal for each of said first and second regions, said pre-processing including performing at least one of;

a source separation with blind source separation, and a constrained source separation;

subtracting said normalized time-series signal corresponding to each of said first and second regions to obtain an environmentally compensated signal; and

communicating said environmentally compensated signal to a computer system.

24. The computer implemented method of claim 23, wherein said pre-processing further comprises, for each channel:

computing an average value of all pixels acquired with this channel to obtain a channel average per image frame;

computing a global channel average and a global standard deviation from said computed averages for this channel;

subtracting said channel average from said global channel average to produce a resulting signal;

dividing said resulting signal by said standard deviation to obtain a zero-mean unit variance time-series signal for said region;

normalizing said time-series signal; and

band-pass filtering said normalized time-series signal to remove frequencies that are above and below expected frequencies of the plethysmographic signals of said subject.

25. The computer implemented method of claim 23, further comprising:

generating a set of reference signals having a frequency range which approximates a frequency range of said subject'"'"'s cardiac pulse;

performing, using said reference signal, a constrained source separation on said source data to obtain an estimated source signal with a minimum error, said minimum error being achieved by adjusting phase of the estimated source signal and calculating a difference between two waveforms; and

estimating a cardiac frequency of said subject based upon a frequency at which said minimum error was achieved.