Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions
First Claim
1. A method of determining heart rate through observation of a human face, comprising:
- acquiring with at least one automated camera, a time series of images of a human face, wherein the time series of images are subject to variations between respective images of the time series in illumination and facial movements;
adaptively selecting, with the at least one automated processor, a subset of the regions of interest of respective images of the time series of images of the human face, that exhibit a more statistically reliable heart-rate-determined variation than a non-selected subset of regions of the respective images of the human face;
based at least on the adaptively selected subset of regions of interest of the respective images of the time series of the human face that exhibit the reliable heart-rate-determined variation, determining a heart rate, and updating the adaptively selected subset of the regions of interest that exhibit the reliable heart-rate-determined variation; and
outputting a signal corresponding to the determined heart rate.
7 Assignments
0 Petitions
Accused Products
Abstract
Recent studies in computer vision have shown that, while practically invisible to a human observer, skin color changes due to blood flow can be captured on face videos and, surprisingly, be used to estimate the heart rate (HR). While considerable progress has been made in the last few years, still many issues remain open. In particular, state-of-the-art approaches are not robust enough to operate in natural conditions (e.g. in case of spontaneous movements, facial expressions, or illumination changes). Opposite to previous approaches that estimate the HR by processing all the skin pixels inside a fixed region of interest, we introduce a strategy to dynamically select face regions useful for robust HR estimation. The present approach, inspired by recent advances on matrix completion theory, allows us to predict the HR while simultaneously discover the best regions of the face to be used for estimation. Thorough experimental evaluation conducted on public benchmarks suggests that the proposed approach significantly outperforms state-of-the-art HR estimation methods in naturalistic conditions.
-
Citations
23 Claims
-
1. A method of determining heart rate through observation of a human face, comprising:
-
acquiring with at least one automated camera, a time series of images of a human face, wherein the time series of images are subject to variations between respective images of the time series in illumination and facial movements; adaptively selecting, with the at least one automated processor, a subset of the regions of interest of respective images of the time series of images of the human face, that exhibit a more statistically reliable heart-rate-determined variation than a non-selected subset of regions of the respective images of the human face; based at least on the adaptively selected subset of regions of interest of the respective images of the time series of the human face that exhibit the reliable heart-rate-determined variation, determining a heart rate, and updating the adaptively selected subset of the regions of interest that exhibit the reliable heart-rate-determined variation; and outputting a signal corresponding to the determined heart rate. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method of determining heart rate from video images, comprising:
-
processing, with the at least one automated processor, a stream of video images of a face from at least one automated camera, to extract a plurality of face regions; computing chrominance features of the plurality of face regions, with at least one automated processor; jointly estimating an underlying low-rank feature matrix and a mask of a selected subset of the plurality of face regions which have a higher statistical reliability than a non-selected subset of the plurality of face regions, using a self-adaptive matrix completion algorithm, with the at least one automated processor; and computing the heart rate from a signal estimate provided by the self-adaptive matrix completion algorithm, with the at least one automated processor. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22)
-
-
23. A system for determining cardiac contraction timing from video images, comprising:
-
an input port configured to receive a time sequence of images of a human face from an automated camera; at least one automated processor, configured to; process the time sequence of images of the human face to extract a plurality of facial regions; compute heartbeat-induced time-varying features of the respective plurality of facial regions; determine a respective statistical parameter for heartbeat-induced time-varying features of the respective plurality of facial regions; adaptively select, based on the determined respective statistical parameter, a dynamically changing subset of the plurality of facial regions having a higher statistical reliability than a non-selected subset; and compute a cardiac contraction timing based on at least the respective heartbeat-induced time-varying features of the respective selected subset of the plurality of facial regions; and an output port configured to convey a signal responsive to the cardiac contraction timing.
-
Specification