COMPENSATING FOR MOTION INDUCED ARTIFACTS IN A PHYSIOLOGICAL SIGNAL EXTRACTED FROM A SINGLE VIDEO

• US 20140376788A1
• Filed: 06/21/2013
• Published: 12/25/2014
• Est. Priority Date: 06/21/2013
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: XU, Beilei, MESTHA, Lalit Keshav, KYAL, Survi, MADHU, Himanshu J.
• Granted Patent: US 9,436,984 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

What is disclosed is a system and method for compensating for motion induce artifacts in a physiological signal obtained from a video. In one embodiment, a video of a first and second region of interest of a subject being monitored for a desired physiological function is captured by a video device. The first region is an area of exposed skin wherein a desired signal corresponding to the physiological function can be registered. The second region is an area where movement is likely to induce motion artifacts into that signal. The video is processed to isolate pixels in the image frames associated with these regions. Pixels of the first region are processed to obtain a time-series signal. A physiological signal is extracted from the time-series signal. Pixels of the second region are analyzed to identify motion. The physiological signal is processed to compensate for the identified motion.

Claims

1. A method for compensating for motion induced artifacts in physiological signals obtained from a single video captured by a single video imaging device of a subject being monitored for a desired physiological function in a non-contact, remote sensing environment, the method comprising:

receiving a video comprising a plurality of time-sequential image frames captured by a video imaging device of at least a first and second region of interest of a subject being monitored for a desired physiological function, said first region being an area of exposed skin wherein a desired signal corresponding to said physiological function can be registered by the video imaging device, said second region being an area where a movement by said subject is likely to induce motion artifacts into said signal;

processing said video to isolate pixels in said image frames associated with each of said first and second regions of interest;

processing pixels of said isolated first region to generate a time-series signal;

extracting, from said time-series signal, a physiological signal corresponding to said desired physiological function;

analyzing pixels of said isolated second region to identify at least one time interval when movement occurred; and

processing said physiological signal over each of said identified time intervals to obtain a processed physiological signal.

2. The method of claim 1, wherein said video device is any of:

a RGB video device, a 3D video device, an infrared video device, a multi-spectral video device, a hyperspectral video device, and a hybrid device comprising any combination hereof.

3. The method of claim 1, wherein said video device is mounted on a robot.

4. The method of claim 1, wherein said first and second regions of interest are any of:

the same region, and overlapping regions.

5. The method of claim 1, wherein said second region of interest is any of:

hands, arms, legs, arm with or without covers.

6. The method of claim 1, wherein said first and second regions of interest are isolated in said image frames using any of:

pixel classification, object identification, facial recognition, color, texture, spatial features, spectral information, pattern recognition, and a user input.

7. The method of claim 1, wherein processing said physiological signal over any given identified time interval comprises any of:

filtering segments of said physiological signal over said given time interval;

discarding segments of said physiological signal over said given time interval where a movement over said given time interval is determined to be above a maximum threshold;

leaving segments of said physiological signal as-is where a movement over said given time interval is determined to be below a minimum threshold;

replacing said segment with a default segment; and

requesting a user selection.

8. The method of claim 1, wherein processing pixels of said isolated first region to generate a time-series signal comprises any of:

determining a center pixel location of said first region by any one of object detection, object tracking, user input;

determining a size and shape of said first region by any one of object detection, object tracking or user input;

weighted averaging of said pixels within said first region in each frame to generate a time series signal, said weights may be equal;

normalizing pixel colors relative to a fixed pixel colors; and

detrending said time-series signal to remove frequency components outside of said physiological signal.

9. The method of claim 1, wherein analyzing pixels of said isolated second region to identify time intervals when a movement occurred is performed by any of:

object tracking, object detection, frame differencing, and a user input.

10. The method of claim 1, wherein analyzing pixels of said isolated second region to identify at least one time interval when movement occurred comprises any of:

a change in center pixel locations of said isolated second region relative to a fixed position;

a change in sizes of said isolated second region relative to a fixed size;

a change in shapes of said isolated second region; and

a residual from frame differencing.

11. The method of claim 1, wherein said desired physiological function is a cardiac function, and said processed physiological signal is a cardiac signal.

12. The method of claim 11, further comprising analyzing said cardiac signal to determine any of:

heart rate variability, cardiac pulse frequency, and pulse transit time.

13. The method of claim 11, further comprising using said cardiac signal to determine an occurrence of any of:

Cardiac Arrhythmia, Cardiac Stress, Cardiac Failure, and Heart Disease.

14. The method of claim 1, wherein said desired physiological function is a respiratory function and said processed physiological signal is a respiratory signal.

15. The method of claim 14, further comprising analyzing said respiratory signal to determine any of:

pulmonary volume, minute ventilation, and respiration rate.

16. The method of claim 14, further comprising using said respiratory signal to determine an occurrence of any of:

Sudden Infant Death Syndrome, Respiratory Distress, Respiratory Failure, and Pulmonary Disease.

17. The method of claim 1, wherein said video is a live streaming video and said processed physiological signal is generated in real-time.

18. The method of claim 1, further comprising communicating said processed physiological signal to any of:

a storage device, a display device, and a remote device over a network.

19. A system for compensating for motion induced artifacts in physiological signals obtained from a single video captured by a single video imaging device of a subject being monitored for a desired physiological function in a non-contact, remote sensing environment, the system comprising:

a video imaging device for capturing video comprising a plurality of time-sequential image frames; and

a processor in communication with a memory and said storage device, said processor executing machine readable program instructions for performing the steps of;

receiving a video captured by said video imaging device of at least a first and second region of interest of a subject being monitored for a desired physiological function, said first region being an area of exposed skin wherein a desired signal corresponding to said physiological function can be registered by the video imaging device, said second region being an area where a movement by said subject is likely to induce motion artifacts into said signal;

processing said video to isolate pixels in said image frames associated with each of said first and second regions of interest;

processing pixels of said isolated first region to generate a time-series signal;

extracting, from said time-series signal, a physiological signal corresponding to said desired physiological function;

analyzing pixels of said isolated second region to identify at least one time interval when movement occurred; and

processing said physiological signal over each of said identified time intervals to obtain a processed physiological signal.

20. The system of claim 19, wherein said video device is any of:

a RGB video device, a 3D video device, an infrared video device, a multi-spectral video device, a hyperspectral video device, and a hybrid device comprising any combination hereof.

21. The system of claim 19, wherein said video device is mounted on a robot.

22. The system of claim 19, wherein said first and second regions of interest are any of:

the same region, and overlapping regions.

23. The system of claim 19, wherein said second region of interest is any of:

hands, arms, legs, arm with or without covers.

24. The system of claim 19, wherein said first and second regions of interest are isolated in said image frames using any of:

pixel classification, object identification, facial recognition, color, texture, spatial features, spectral information, pattern recognition, and a user input.

25. The system of claim 19, wherein processing said physiological signal over any given identified time interval comprises any of:

filtering segments of said physiological signal over said given time interval;

discarding segments of said physiological signal over said given time interval where a movement over said given time interval is determined to be above a maximum threshold;

leaving segments of said physiological signal as-is where a movement over said given time interval is determined to be below a minimum threshold;

replacing said segment with a default segment; and

requesting a user selection.

26. The system of claim 19, wherein processing pixels of said isolated first region to generate a time-series signal comprises any of:

determining a center pixel location of said first region by any one of object detection, object tracking, user input;

determining a size and shape of said first region by any one of object detection, object tracking or user input;

weighted averaging of said pixels within said first region in each frame to generate a time series signal, said weights may be equal;

normalizing pixel colors relative to a fixed pixel colors; and

detrending said time-series signal to remove frequency components outside of said physiological signal.

27. The system of claim 19, wherein analyzing pixels of said isolated second region to identify time intervals when a movement occurred is performed by any of:

object tracking, object detection, frame differencing, and a user input.

28. The system of claim 19, wherein analyzing pixels of said isolated second region to identify at least one time interval when movement occurred comprises any of:

a change in center pixel locations of said isolated second region relative to a fixed position;

a change in sizes of said isolated second region relative to a fixed size;

a change in shapes of said isolated second region; and

a residual from frame differencing.

29. The system of claim 19, wherein said desired physiological function is a cardiac function, and said processed physiological signal is a cardiac signal.

30. The system of claim 29, further comprising analyzing said cardiac signal to determine any of:

heart rate variability, cardiac pulse frequency, and pulse transit time.

31. The system of claim 29, further comprising using said cardiac signal to determine an occurrence of any of:

Cardiac Arrhythmia, Cardiac Stress, Cardiac Failure, and Heart Disease.

32. The system of claim 19, wherein said desired physiological function is a respiratory function and said processed physiological signal is a respiratory signal.

33. The system of claim 32, further comprising analyzing said respiratory signal to determine any of:

pulmonary volume, minute ventilation, and respiration rate.

34. The system of claim 32, further comprising using said respiratory signal to determine an occurrence of any of:

Sudden Infant Death Syndrome, Respiratory Distress, Respiratory Failure, and Pulmonary Disease.

35. The system of claim 19, wherein said video is a live streaming video and said processed physiological signal is generated in real-time.

36. The system of claim 19, further comprising communicating said processed physiological signal to any of:

a storage device, a display device, and a remote device over a network.