System and method for extracting physiological information from remotely detected electromagnetic radiation

• US 10,441,173 B2
• Filed: 12/20/2013
• Issued: 10/15/2019
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Shan, Caifeng, Dubielczyk, Alexander, Schlack, Andreas Wolfgang, Neumann, Rolf
• Primary Examiner(s): Gillman, Amelie R
• Application Number:
  US14/135,636
• Publication Number:
  US 20140180132A1
• Time in Patent Office: 2,125 Days
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/02 Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 7/0012 Biomedical image inspection

Abstract

A system and a related method extract physiological information from remotely detected electromagnetic radiation. The system includes an interface configured for receiving a data stream including image data representing an observed overall region of at least one subject of interest. A partitioning unit defines a plurality of sub regions in the overall region. A classifier classifies the plurality of sub regions into at least one indicative type of region and at least one auxiliary type of region. The at least one indicative type of region includes at least one indicative region of interest at least partially representing the subject of interest. The at least one auxiliary type of region includes at least one reference region. The system further comprises a data processor configured for processing at least one sub region classified as region of interest, thereby obtaining vital information.

Claims

1. A system for monitoring at least one subject of interest to extract physiological information from remotely detected electromagnetic radiation re-emitted by the at least one subject of interest, comprising:

an interface configured for receiving a data stream comprising image data representing an observed overall region comprising the at least one subject of interest;

a display device;

one or more processors configured to;

partition the overall region into a first plurality of sub regions of the overall region;

classify the first plurality of sub regions as (1) a first plurality of sub regions of interest, image data of the data stream corresponding to the first plurality of sub regions of interest including image data to be analyzed to determine a vital sign and being artifacted by noise, motion, and/or illumination variations, and (2) a first plurality of reference sub regions, image data of the data stream corresponding to the first plurality of reference sub regions being indicative of the noise, the motion, and/or the illumination variances and not indicative of the vital sign, wherein classifying the first plurality of sub regions further includes identifying at least one of the first plurality of sub regions as a first tracking sub region, wherein the first tracking sub region is indicative of a first characteristic feature of the subject, rank the first plurality of sub regions of interest;

select at least one first sub region of interest based on the rank;

correct the image data of the data stream corresponding to the selected at least one first sub region of interest based on the image data of the data stream corresponding to at least one of the first plurality of reference sub regions;

determine the vital sign from the corrected image data from the selected at least one first sub region of interest;

maintain a positional or spatial offset between the selected at least one first sub region of interest and the first tracking sub region;

over a course of monitoring the subject, in response to quality check values indicative of disturbances or faults exceeding predefined thresholds or a quality and/or accuracy parameter being out of range, partition the overall region into a second plurality of sub regions,

classify the second plurality of sub regions as (1) a second plurality of sub regions of interest, image data of the data stream corresponding to the second plurality of sub regions of interest including image data to be analyzed to determine the vital sign and being artifacted by the noise, the motion, and/or the illumination variations, and (2) a second plurality of reference sub regions, image data of the data stream corresponding to the second plurality of reference sub regions being indicative of the noise, the motion, and/or the illumination variances and not indicative of the vital sign, wherein classifying the second plurality of sub regions further includes identifying at least one of the second plurality of sub regions as a second tracking sub region, wherein the second tracking sub region is indicative of a second characteristic feature of the subject,

rank the second plurality of sub regions of interest;

select at least one second sub region of interest based on the rank;

correct the image data of the data stream corresponding to the selected at least one second sub region of interest based on the image data of the data stream corresponding to at least one of the second plurality of reference sub regions;

determine the vital sign from the corrected image data from the at least one selected second sub region of interest;

maintain a positional or spatial offset between the selected at least one second sub region of interest and the second tracking sub region; and

control the display device to display the determined vital sign.

2. The system as claimed in claim 1, wherein the one or more processors is further configured to:

rank the first plurality of reference sub regions, and correct the image data of the data stream corresponding to the selected at least one first sub region of interest based on the image data of the data stream corresponding to a highest ranked first plurality of reference sub regions;

or

rank the second plurality of reference sub regions, and correct the image data of the data stream corresponding to the selected at least one second sub region of interest based on the image data of the data stream corresponding to a highest ranked second plurality of reference sub regions.

3. The system as claimed in claim 1, wherein classifying the first and second pluralities of sub regions includes classifying the first and second pluralities of sub regions according to a classification scheme, wherein the classification scheme comprises at least one classification parameter selected from the group consisting of color model match, feature presence, image contrast, illumination condition, spatial or temporal illumination variation, reflectance, anatomic location, body part presence, vital information accuracy, vital information reliability, and combinations thereof.

4. The system as claimed in claim 1, wherein the data stream comprises at least one channel of image data containing depth-representative information and wherein the one or more processors are further configured to:

generate a relief image of at least a portion of the overall region,

identify a prominent feature of the subject from the relief image,

track motion within the overall region based on motion of the prominent feature.

5. The system as claimed in claim 1, wherein the data stream comprises at least two channels of image data representing different wavelength ranges including a visible wavelength range and a wavelength range that penetrates skin of the at least one subject of interest.

6. A method for extracting physiological information from remotely detected electromagnetic radiation, comprising, with one or more processors:

receiving a data stream comprising image data representing an observed overall region comprising a subject of interest;

partitioning the overall region into a plurality of sub regions in a selected pattern; and

classifying the plurality of sub regions as;

sub regions of interest, wherein image data of the data stream corresponding to the sub regions of interest vary in intensity with a vital sign,

reference sub regions, wherein image data of the data stream corresponding to the reference sub regions vary with illumination and do not vary with the vital sign, and

a tracking portion of the overall region, wherein image data of the data stream corresponding to the tracking portion vary with movement of the subject;

ranking the sub regions classified as the sub regions of interest;

selecting at least one sub region of interest based on the ranking of the sub regions of interest;

maintaining a positional or spatial offset between the selected at least one region of interest and a characteristic feature of the tracking portion;

correcting image data of the data stream corresponding to the at least one selected sub region of interest based on the image data of the data stream corresponding to at least one of the reference sub regions and the tracking portion;

determining a vital sign from the corrected image data;

controlling a display device to display the determined vital sign;

detecting at least one quality or accuracy parameter and in response to the detected at least one quality or accuracy parameter being beyond a selected range, re-partitioning the overall region into re-partitioned sub regions;

classifying the re-partitioned sub regions;

updating the vital sign based on the re-partitioned sub regions; and

controlling the display device to display the updated vital sign.

7. The method as claimed in claim 6, wherein determining the vital sign includes with the one or more processors:

determining a plurality of sub region vital signs based on the corrected image data; and

combining the determined sub region vital signs to obtain the vital sign, wherein the combining comprises averaging, weighted averaging, and/or taking a median.

8. A non-transitory computer-readable medium which carries software configured to control a computer to carry out the steps of the method as claimed in claim 6.

9. A photoplethysmographic system comprising:

an interface configured to receive an image data stream from an electromagnetic radiation imaging sensor, the image data stream representing an observed overall region including a subject of interest;

one or more processors configured to;

(a) partition the overall region into sub regions, a component of the image data stream corresponding to each of the sub regions;

(b) classify the sub regions as (1) sub regions of interest and (2) reference sub regions, image data stream components corresponding to the sub regions of interest varying with a vital sign of the subject and noise and changes in lighting, and image data stream components corresponding to the reference sub regions varying with the noise and the lighting;

(c) correct at least one of the sub region of interest components of the image data stream for the noise and the changes in lighting with at least one of the image data stream components corresponding to the reference sub regions;

(d) determine a vital sign from the corrected at least one sub region of interest component of the image data stream;

(e) generate a relief image of a portion of the overall region;

(f) monitor a prominent feature in the relief image for motion;

(g) maintain a spatial offset between the prominent feature and the corrected at least one sub region of interest;

(h) in response to detecting that at least one quality or accuracy parameter is beyond a selected range, return to processor operation (a) and repeat at least operations (a), (b), (c), and (d); and

a display controlled by the one or more processors to display the determined vital sign.

10. The system as claimed in claim 9, wherein the corrected at least one of the sub region of interest components of the image data stream comprises a plurality of corrected sub regions of interest components of the image data stream;

and wherein the one or more processors is further configured to;

rank the corrected sub region of interest components of the image data stream, and select a highest ranked corrected sub region of interest component of the image data stream;

wherein the vital sign is determined based on the highest ranked corrected sub region of interest data stream component.