Physiological signals measurement systems and methods thereof

• US 10,383,531 B2
• Filed: 12/19/2017
• Issued: 08/20/2019
• Est. Priority Date: 08/09/2017
• Status: Active Grant

• Current Assignee: Wistron Corporation
• Original Assignee: Wistron Corporation
• Inventors: Lai, Yu-Ren, Chen, Yu-Yen
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US15/847,800
• Publication Number:
  US 20190046057A1
• Time in Patent Office: 609 Days
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/7278 Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 2207/30104 Vascular flow; Blood flow; Perfusion
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images

Abstract

A method for measuring a physiological signal is provided, including the steps of: receiving a video and performing face detection for each image frame thereof; performing photo-plethysmography (PPG) calculation and analysis on first face image for first image frame to obtain first PPG information of first regions and second PPG information of corresponding second regions; determining at least one region-of-interest (ROI) and one noise reference region according to the first PPG information and the second PPG information; generating ROI information and noise reference region information based on the ROI and the noise reference region; and performing the PPG calculation and analysis on the ROI and the noise reference region of the second face image of each subsequent second image frame and generating corresponding third PPG information; and counting the PPG information of all the image frames to calculate a measured value of a physiological signal.

Claims

1. A method for measuring a physiological signal, comprising:

receiving a video including a plurality of continuous image frames and performing face detection on each of the image frames to obtain a face image for each image frame, wherein the continuous image frames include a first image frame and a plurality of second image frames, and wherein the first image frame includes a first face image having first regions, each of the first regions having a plurality of second regions corresponding thereto;

performing a photo-plethysmography (PPG) calculation and analysis on the first face image of the first image frame to obtain a plurality of first PPG information of the first regions and a plurality of second PPG information of corresponding second regions;

determining a reserved region from the first regions according to the first PPG information of the first regions and a first threshold value, and determining at least one region-of-interest (ROI) and a noise reference region according to results of comparing the second PPG information of corresponding second regions for the reserved region and a second threshold value;

generating an ROI information and a noise reference region information based on the at least one ROI and the noise reference region of the first face image, wherein the ROI information includes location information and the PPG information corresponding to the at least one ROI and the noise reference region information includes location information and the PPG information corresponding to the noise reference region;

performing the PPG calculation and analysis on the ROI and the noise reference region of each second face image of each of the second image frames and generating corresponding third PPG information based on the ROI information and the noise reference region information; and

counting the PPG information of all the image frames to calculate a measured value of the physiological signal.

2. The method for measuring the physiological signal of claim 1, further comprising:

performing a rough skin segmentation on skin portion of the first face image according to face feature points to divide the first face image into the first regions.

3. The method for measuring the physiological signal of claim 1, wherein the step of determining the reserved region from the first regions according to the first PPG information of the first regions and the first threshold value further comprises:

comparing the first PPG information of each of the first regions with the first threshold value; and

setting one of the first regions whose first PPG information is within a range of the first threshold value as the reserved region.

4. The method for measuring the physiological signal of claim 3, wherein the step of determining the at least one ROI and the noise reference region according to the results of comparing the second PPG information of corresponding second regions for the reserved region and the second threshold value further comprises:

comparing the second PPG information of each of the second regions with the second threshold value; and

setting one of the second regions whose second PPG information is within a range of the second threshold value as the at least one ROI and determining the noise reference region from the second regions whose second PPG information are not within the range of the second threshold value.

5. The method for measuring the physiological signal of claim 4, wherein the step of determining the noise reference region from the second regions whose second PPG information are not within the range of the second threshold value further comprises:

performing a variance calculation on all the second regions whose second PPG information are not within the range of the second threshold value to obtain a plurality of variance values; and

determining the noise reference region based on the variance values;

wherein the noise reference region is a second region having the minimum variation value among the second regions whose second PPG information are not within the range of the second threshold value.

6. The method for measuring the physiological signal of claim 1, wherein the step of performing the PPG calculation and analysis on the first face image of the first image frame to obtain the first PPG information of the first regions and the second PPG information of corresponding second regions further comprises:

performing a pulse detection algorithm on each of the first regions and the corresponding second regions to perform the PPG calculation and analysis to obtain the first PPG information of the first region and the second PPG information of corresponding second regions.

7. The method for measuring the physiological signal of claim 1, further comprising:

performing the face detection to calculate a difference in the face position and size between currently processed image frame and a previously processed image frame;

directly performing the PPG calculation and analysis on the ROI of the currently processed image frame and obtaining the PPG information of the currently processed image frame based on the PPG calculation result when the difference is not greater than a threshold value; and

setting the currently processed image frame as the first image frame to redefine the ROI and the noise reference region when the difference is greater than the threshold value.

8. A physiological signal measurement system, comprising:

an image input device, configured to obtain a video including a plurality of continuous image frames, wherein the continuous image frames include a first image frame and a plurality of second image frames, and wherein the first image frame includes a first face image having first regions, each of the first regions having a plurality of second regions corresponding thereto;

an image processor coupled to the image input device, configured to receive the video and perform face detection on each of the image frames to obtain a face image for each image frame, perform a photo-plethysmography (PPG) calculation and analysis on the first face image of the first image frame to obtain a plurality of first PPG information of the first regions and a plurality of second PPG information of the corresponding second regions, determine a reserved region from the first regions according to the first PPG information of the first regions and a first threshold value, determines at least one region-of-interest (ROI) and a noise reference region according to results of comparing the second PPG information of corresponding second regions for the reserved region and a second threshold value, generate an ROI information and a noise reference region information based on the at least one ROI and the noise reference region of the first face image, wherein the ROI information includes location information and the PPG information corresponding to the at least one ROI and the noise reference region information includes location information and the PPG information corresponding to the noise reference region, perform the PPG calculation and analysis on the ROI and the noise reference region of each second face image of each of the second image frames and generate corresponding third PPG information based on the ROI information and the noise reference region information, and count the PPG information of all the image frames to calculate a measured value of a physiological signal; and

a display device coupled to the image processor, configured to display the measured value of the physiological signal.

9. The physiological signal measurement system of claim 8, wherein the image processor further performs a rough skin segmentation on skin portion of the first face image according to face feature points to divide the first face image into the first regions.

10. The physiological signal measurement system of claim 8, wherein the image processor further compares the first PPG information of each of the first regions with the first threshold value and sets one of the first regions whose first PPG information is within a range of the first threshold value as the reserved region.

11. The physiological signal measurement system of claim 10, wherein the image processor further compares the second PPG information of each of the second regions with the second threshold value, sets one of the second regions whose second PPG information is within a range of the second threshold value as the at least one ROI, and determines the noise reference region from the second regions whose second PPG information are not within the range of the second threshold value.

12. The physiological signal measurement system of claim 11, wherein the image processor further performs a variance calculation on all the second regions whose second PPG information are not within the range of the second threshold value to obtain a plurality of variance values and determines the noise reference region based on the variance values, wherein the image processor sets a second region having the minimum variation value among the second regions whose second PPG information are not within the range of the second threshold value as the noise reference region.

13. The physiological signal measurement system of claim 8, wherein the image processor performs a pulse detection algorithm on each of the first regions and the corresponding second regions to perform the PPG calculation and analysis to obtain the first PPG information of the first region and the second PPG information of corresponding second regions.

14. The physiological signal measurement system of claim 8, wherein the image processor further performs the face detection to calculate a difference in the face position and size between currently processed image frame and a previously processed image frame, directly performs the PPG calculation and analysis on the ROI of the currently processed image frame and obtain the PPG information of the currently processed image frame based on the PPG calculation result when the difference is not greater than a threshold value, and sets the currently processed image frame as the first image frame to redefine the ROI and the noise reference region when the difference is greater than the threshold value.