Pulse detection from head motions in video

• US 10,638,942 B2
• Filed: 06/25/2014
• Issued: 05/05/2020
• Est. Priority Date: 06/26/2013
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Balakrishnan, Guha, Guttag, John V., Durand, Frederic
• Primary Examiner(s): Dietrich, Joseph M
• Application Number:
  US14/315,174
• Publication Number:
  US 20150005646A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/004 adapted for image acquisition of a particular organ or body part A61B5/0082 takes precedence; arrangements for optical scanning A61B5/0062
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0255 Recording instruments specially adapted therefor
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence

Abstract

Heart rates and beat lengths can be extracted from videos by measuring subtle head motion caused by the Newtonian reaction to the influx of blood at each beat. In an embodiment of the present invention, a method tracks features on the head and performs principal component analysis (PCA) to decompose their trajectories into a set of component motions. The method then selects a component that best corresponds to heartbeats based on its temporal frequency spectrum. Finally, the motion projected to this component is analyzed and peaks of the trajectories are identified, which correspond to heartbeats.

Claims

1. A method comprising:

tracking trajectories of respective feature points within a selected region of a digital video by;

determining a maximum distance of movement of each feature point between consecutive frames of the digital video; and

discarding feature points corresponding to points having a maximum distance between consecutive frames of the digital video exceeding a mode of distribution;

extracting a plurality of dimensional components from each tracked trajectory of the respective feature points within the selected region of the digital video; and

determining an oscillation rate of a subject represented by the feature points of the digital video by analyzing the dimensional components of the tracked trajectories of the respective feature points within the selected region of the digital video.

2. The method of claim 1, wherein the oscillation rate is a heart rate or pulse rate of the subject.

3. The method of claim 1, wherein the plurality of extracted components includes a vertical component, horizontal component, or vertical and horizontal components.

4. The method of claim 1, wherein selecting the region of the digital video includes finding a region of interest including a head of the subject.

5. The method of claim 1, wherein analyzing the motion of the feature points includes:

filtering the extracted feature points temporally;

generating a plurality of components using principal component analysis on the temporally filtered extracted feature points;

selecting one of the components of the plurality of components generated by the principal component analysis;

detecting peaks of the selected component; and

determining the oscillation rate based on the detected peaks of the selected component.

6. The method of claim 5, wherein filtering the extracted feature points temporally removes frequencies that are outside a predetermined range of pulse rates.

7. The method of claim 5, wherein performing principal component analysis includes decomposing the filtered feature points into a set of independent source signals.

8. The method of claim 5, wherein selecting the component includes selecting the component having a clearest main frequency.

9. A system comprising:

a processor; and

a memory with computer code instructions stored therein, the memory operatively coupled to said processor such that the computer code instructions configure the processor to implement;

a tracking module configured to track trajectories of respective feature points within a selected region of a digital video by;

determining a maximum distance of movement of each feature point between consecutive frames of the digital video; and

discarding feature points corresponding to points having a maximum distance between consecutive frames of the digital video exceeding a mode of distribution;

an extraction module configured to extract a plurality of dimensional components from each tracked trajectory of the respective feature points within the selected region of the digital video; and

an analysis module configured to determine an oscillation rate of a subject represented by the feature points of the digital video by analyzing the dimensional components of the tracked trajectories of the respective feature points within the selected region of the video.

10. The system of claim 9, wherein the oscillation rate is a heart rate or pulse rate of the subject.

11. The system of claim 9, wherein the extracted component is a vertical component, horizontal component, or vertical and horizontal component.

12. The system of claim 9, wherein the selected region is selected by a selection module implemented by the processor, and the selection module is further configured to find a region of interest including a head of the subject.

13. The system of claim 9, wherein the analysis module is further configured to

filter the extracted feature points temporally;

generate a plurality of components using principal component analysis on the temporally filtered extracted feature points;

select one of the components of the plurality of components generated by the principal component analysis;

detect peaks of the selected component; and

determine the oscillation rate based on the detected peaks of the selected component.

14. The system of claim 13, wherein the analysis module is further configured to filter the extracted feature points temporally by removing frequencies that are outside a predetermined range of pulse rates.

15. The system of claim 13, wherein the analysis module is further configured to perform principal component analysis by decomposing the filtered feature points into a set of independent source signals.

16. The system of claim 13, wherein the analysis module is further configured to select the component by selecting the component having a clearest main frequency.

17. A non-transitory computer readable medium having instructions stored thereon, the instructions configured to, when loaded and executed by a processor, cause the processor to:

track trajectories of respective feature points within a selected region of a digital video by;

determining a maximum distance of movement of each feature point between consecutive frames of the digital video; and

discarding feature points corresponding to points having a maximum distance between consecutive frames of the digital video exceeding a mode of distribution;

extract a plurality of dimensional components from each tracked trajectory of the respective feature points within the selected region of the digital video; and

determine an oscillation rate of a subject represented by the feature points of the digital video by analyzing the dimensional components of the tracked trajectories of the respective feature points within the selected region of the digital video.