Compensating for motion during real-time batch processing of video for physiological function assessment

• US 9,483,837 B2
• Filed: 03/14/2014
• Issued: 11/01/2016
• Est. Priority Date: 03/14/2014
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Xu, Beilei, Kyal, Survi, Mestha, Lalit Keshav, Pennington, Graham S.
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Wang, Carol
• Application Number:
  US14/211,455
• Publication Number:
  US 20150262338A1
• Time in Patent Office: 963 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

What is disclosed is a system and method for compensating for motion during processing of a video of a subject being monitored for physiological function assessment. In one embodiment, image frames are received. Successive batches of N video frames are processed to isolate pixels associated with a body region of the subject where a physiological signal is registered by the camera. The pixels are processed to obtain a time-series signal for each batch. A determination is made whether movement during video acquisition of this batch of image frames exceeds a threshold level. If so then a size N of the next batch of image frames is changed to: N=N+M₁, where N+M₁≦N_max. Otherwise, a size N of a next batch is changed to: N=N−M₂, where N−M₂≧N_min. Thereafter, processing repeats in a real-time continuous manner as the next batch of the N image frames is received. Various embodiments are disclosed.

Claims

1. A method for compensating for motion during processing of a video of a subject being monitored for a physiological function, the method comprising:

receiving image frames of a video captured by a video imaging device of a subject being monitored for a desired physiological function, said image frames being acquired of at least a portion of a body region of said subject where a physiological signal corresponding to said physiological function is registered by said video imaging device used to capture said video;

defining an initial size N of a batch of image frames for processing such that N_min≦

N≦

N_max, where N_min is a minimum size of a batch of image frames and N_max is a maximum size of a batch of image frames;

defining a number of frames M₁ by which a size of a given batch of image frames is to be increased, and a number of frames M₂ by which a size of a given batch of image frames is to be decreased, where M₁, M₂≧

1; and

for a successive batch of image frames of size N;

processing a current batch of image frames to isolate pixels associated with said body region;

processing said isolated pixels to obtain a time-series signal for the current batch;

determining whether a movement occurred during video acquisition of the current batch of image frames; and

in response to the movement exceeding a threshold level, providing an indication that the time-series signal obtained from the current batch of image frames for processing may be unreliable and increasing a size of a next batch of image frames to;

N=N+M₁, where N+M₁≦

N_max, otherwise decreasing a size of a next batch of image frames to;

N=N−

M₂, where N−

M₂≧

N_min, wherein the size of a next batch of image frames is increased or decreased by using data from previous successive batches of image frames.

2. The method of claim 1, where said video imaging device is any of:

a color video camera, a monochrome video camera, an infrared video camera, a multispectral video imaging device, and a hyperspectral video camera.

3. The method of claim 1, wherein pixels associated with said body region are isolated in said image frames using any of:

pixel classification based on color, texture, spatial features, spectral information, object identification, pattern recognition, and a user input.

4. The method of claim 1, wherein determining whether said movement occurred comprises any of:

determining a rate of change of said time series signal of at least one pre-selected channel of said video camera;

determining whether a derivative of any X or Y locations, orientations, size in said region of interest has changed relative to a previous time; and

determining whether a derivative of any X or Y locations in other parts of the subject'"'"'s body exceed a pre-defined threshold.

5. The method of claim 1, wherein said threshold level of movement is based upon any of:

said physiological function for which said subject is being monitored, a type of motion detected in said batch of video image frames, a source of said motion, the time of motion within the batch, and a user input.

6. The method of claim 1, further comprising, on a successive iteration, dynamically adjusting an amount of any of:

M₁ and M₂, based on an amount said movement exceeded said threshold.

7. The method of claim 1, wherein, in response to said movement exceeding said threshold, further comprising any of:

discarding said time-series signal obtained from the current batch, and applying motion compensation strategies to reduce the effect of motion.

8. The method of claim 1, wherein, in response to said movement exceeding said threshold level, further comprising any of:

initiating an alert signal, signaling a medical professional, changing the frame rate of said video imaging device, moving a position of said video imaging device, swapping the video imaging device for another video camera, and stopping video acquisition of said subject.

9. The method of claim 1, further comprising processing said time-series signal to obtain a physiological signal which corresponds to said physiological function.

10. The method of claim 9, further comprising communicating said physiological signal to any of:

a display device, a storage device, a handheld wireless cellular device, and a remote device over a network.

11. The method of claim 1, further comprising dynamically adjusting said threshold level of movement in real-time based on any of:

a signal-to-noise ratio of a power spectrum of said time-series signal, a waveform of a rhythm signal, and a user input.

12. The method of claim 1, wherein said movement was caused by any of:

a movement by said subject, a movement by said camera, a third party, and by the environment surrounding said subject.

13. The method of claim 1, wherein said video is a streaming video and said video image frames are processed in real-time as they are received.

14. A system for compensating for motion during processing of a video of a subject being monitored for a physiological function, the system comprising:

a memory and a storage device; and

a processor in communication with said memory and storage device, said processor executing machine readable instructions for performing;

receiving image frames of a video captured by a video imaging device of a subject being monitored for a desired physiological function, said image frames being acquired of at least a portion of a body region of said subject where a physiological signal corresponding to said physiological function is registered by said video imaging device used to capture said video;

defining an initial size N of a batch of image frames for processing such that N_min≦

N≦

N_max, where N_min is a minimum size of a batch of image frames and N_max is a maximum size of a batch of image frames;

defining a number of frames M₁ by which a size of a given batch of image frames is to be increased, and a number of frames M₂ by which a size of a given batch of image frames is to be decreased, where M₁, M₂≧

1; and

for a next batch of image frames of size N;

processing a current batch of image frames to isolate pixels associated with said body region;

processing said isolated pixels to obtain a time-series signal for the current batch;

determining whether a movement occurred during video acquisition of the current batch of image frames; and

in response to said movement exceeding a threshold level, providing an indication that the time-series signal obtained from the current batch of image frames may be unreliable and increasing a size of a next batch of image frames to;

N=N+M₁, where N+M₁≦

N_max, otherwise decreasing a size of a next batch of image frames to;

N=N−

M₂, where N−

M₂≧

N_min, wherein the size of a next batch of image frames is increased or decreased by using data from previous successive batches of image frames.

15. The system of claim 14, where said video imaging device is any of:

a color video camera, a monochrome video camera, an infrared video camera, a multispectral video imaging device, and a hyperspectral video camera.

16. The system of claim 14, wherein pixels associated with said body region are isolated in said image frames using any of:

pixel classification based on color, texture, spatial features, spectral information, object identification, pattern recognition, and a user input.

17. The system of claim 14, wherein determining whether said movement occurred comprises any of:

determining a rate of change of said time series signal of at least one pre-selected channel of said video camera;

determining whether a derivative of any X or Y locations, orientations, size in said region of interest has changed relative to a previous time; and

determining whether a derivative of any X or Y locations in other parts of the subject'"'"'s body exceed a pre-defined threshold.

18. The system of claim 14, wherein said threshold level of movement is based upon any of:

said physiological function for which said subject is being monitored, a type of motion detected in said batch of video image frames, a source of said motion, the time of motion within the batch, and a user input.

19. The system of claim 14, further comprising, on a successive iteration, dynamically adjusting an amount of any of:

M₁ and M₂, based on an amount said movement exceeded said threshold.

20. The system of claim 14, wherein, in response to said movement exceeding said threshold, further comprising any of:

discarding said time-series signal obtained from the current batch, and applying motion compensation strategies to reduce the effect of motion.

21. The system of claim 14, wherein, in response to said movement exceeding said threshold level, further comprising any of:

initiating an alert signal, signaling a medical professional, changing the frame rate of said video imaging device, moving a position of said video imaging device, swapping the video imaging device for another video camera, and stopping video acquisition of said subject.

22. The system of claim 14, further comprising processing said time-series signal to obtain a physiological signal which corresponds to said physiological function.

23. The system of claim 22, further comprising communicating said physiological signal to any of:

a display device, a storage device, a handheld wireless cellular device, and a remote device over a network.

24. The system of claim 14, further comprising dynamically adjusting said threshold level of movement in real-time based on any of:

a signal-to-noise ratio of a power spectrum of said time-series signal, a waveform of a rhythm signal, and a user input.

25. The system of claim 14, wherein said video is a streaming video and said video image frames are processed in real-time as they are received.