Device, system and method for detecting apnoea of a subject

• US 10,524,725 B2
• Filed: 06/04/2015
• Issued: 01/07/2020
• Est. Priority Date: 06/06/2014
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Jeanne, Vincent
• Primary Examiner(s): Natnithithadha, Navin
• Application Number:
  US15/315,396
• Publication Number:
  US 20170119304A1
• Time in Patent Office: 1,678 Days
• Field of Search: 600481, 600483, 600484, 600509, 600513, 600529, 600534, 600595
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/113 occurring during breathing
• CPC Class Codes: A61B 5/1135 by monitoring thoracic expansion
• CPC Class Codes: A61B 5/4818 Sleep apnoea
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity

Abstract

A device, system and method unobtrusively and reliably detect apnoea of a subject (2). An input unit (11) receives image data of subject. The image data includes a sequence of images over time. A cardiac activity extraction unit (12) extracts from the image data a cardiac activity signal representing the subject'"'"'s cardiac activity from a skin area of the subject using remote photoplethysmography. A motion signal extraction unit (14) extracts from said image data a motion signal representing motion of a subject'"'"'s body part caused by breathing of the subject. An analysis unit (16) determines a similarity between said cardiac activity signal and said motion signal. A decision unit (18) detects apnoea of the subject based on the determined similarity.

Claims

1. A medical monitor device for detecting apnoea of a subject, comprising:

a video camera configured to monitor the subject and generate image data indicative thereof, said image data including a sequence of images over time,

an input unit for receiving the image data of subject from the video camera,

a cardiac activity extraction unit for extracting from said image data a cardiac activity signal representing the subject'"'"'s cardiac activity from a skin area of the subject using remote photoplethysmography,

a motion signal extraction unit for extracting from said image data a motion signal representing motion of a subject'"'"'s body part caused by breathing of the subject,

an analysis unit for determining the similarity between said cardiac activity signal and said motion signal, and

a decision unit for detecting apnoea of the subject based on the determined similarity, and generating an apnoea warning signal in response to detecting the apnoea,

wherein the motion signal represents motion of the subject'"'"'s body part caused by the breathing of the subject and motion of the subject'"'"'s body part caused by cardiac activity of the subject such that the motion signal has a fundamental frequency of a respiratory rate of the subject when the subject is breathing and has a fundamental frequency of a heart rate of the subject during apnoea, and wherein

during apnoea, the cardiac activity signal and the motion signal have a higher similarity, and

when the subject is breathing normally, the similarity between the cardiac activity signal and the motion signal is lower.

2. The medical monitor device as claimed in claim 1,

wherein the analysis unit is configured to determine a similarity measure representing said similarity, and

wherein the decision unit is configured to detect apnoea by comparing said similarity measure against a predetermined threshold and/or earlier determined similarity measures.

3. The medical monitor device as claimed in claim 1, further comprising:

a movement detection unit for detecting movement of at least a part of the subject by use of said image data,

wherein the decision unit is configured to take detected movement into account in the detection of apnoea of the subject based on the determined similarity.

4. The medical monitor device as claimed in claim 1,

wherein the input unit is configured to receive movement information representing movement of at least a part of the subject,

wherein the decision unit is configured to take the received movement information into account in the detection of apnoea of the subject based on the determined similarity.

5. The medical monitor device as claimed in claim 1, further comprising:

a measurement area detection unit for detecting in said image data a skin area for extraction of said cardiac activity signal and a body part area for extraction of said motion signal.

6. The medical monitor device as claimed in claim 5,

wherein said measurement area detection unit is configured to detect the skin area and a body part area by segmenting the images.

7. The medical monitor device as claimed in claim 1,

wherein the analysis unit is configured to determine the similarity by determining the correlation between said cardiac activity signal and said motion signal over a time window.

8. The medical monitor device as claimed in claim 1,

wherein the analysis unit is configured to determine the similarity by comparing fundamental frequencies present in said cardiac activity signal and said motion signal.

9. The medical monitor device as claimed in claim 1, comprising:

a motion sensor for acquiring motion information representing motion of at least a part of the subject.

10. The medical monitor device as claimed in claim 1,

wherein the motion signal represents motion of the subject'"'"'s body part caused by the breathing of the subject and motion of the subject'"'"'s body part caused by cardiac activity of the subject, such that when the subject stops breathing, the motion signal represents cardiac activity.

11. A medical monitor for detecting apnoea of a subject, comprising:

a video camera for generating image data of the subject, said image data including a sequence of images over time,

an input unit for receiving the image data,

a cardiac activity extraction unit for extracting from the received image data a cardiac activity signal representing cardiac activity of the subject from a skin area of the subject using remote photoplethysmography,

a motion signal extraction unit for extracting from said received image data a motion signal representing motion of a subject'"'"'s body part caused by breathing of the subject,

an analysis unit configured to determine a similarity between said cardiac activity signal and said motion signal by;

comparing fundamental frequencies present in said cardiac activity signal and said motion signal, and

determining a Euclidian distance between the fundamental frequencies, a ratio between the fundamental frequencies and/or an energy ratio of the fundamental frequencies, in particular at the heart rate frequency, and

a decision unit for detecting apnoea of the subject based on the determined similarity and outputting an apnoea warning in response to detecting the apnoea.

12. A method of monitoring for apnoea in a subject, comprising:

receiving video image data of subject, said image data including a sequence of images over time, the images of the sequence each including (i) a skin area which changes color with cardiac activity and (ii) a body region which moves with respiration and cardiac motion,

extracting from said video image data a cardiac activity signal representing the subject'"'"'s cardiac activity from the skin area of the subject using remote photoplethysmography,

extracting from said video image data a motion signal representing motion of the body region caused by breathing and cardiac activity of the subject, wherein during apnoea, the cardiac activity dominates the motion signal,

determining a similarity measure between said cardiac activity signal and said motion signal indicating occurrence of apnoea of the subject when the similarity measure exceeds a predetermined threshold,

in response to the motion signal indicating the occurrence of apnoea, generating an apnoea warning,

wherein the motion signal represents motion of the subject'"'"'s body part caused by the breathing of the subject and motion of the subject'"'"'s body part caused by cardiac activity of the subject, such that when the subject stops breathing, the motion signal represents cardiac activity.

13. A non-transitory computer-readable medium carrying a computer program comprising program code configured to control a computer to carry out the steps of the method as claimed in claim 12 when said computer program is carried out on a computer.

14. A monitor for monitoring a subject for apnoea, comprising:

a camera configured to generate a sequence of images of at least a skin area and a body part area of the subject over time;

one or more computer processors configured to;

from a first portion of the images of the sequence corresponding to the skin area of the subject, extracting a cardiac activity signal using remote photoplethysmography, the cardiac activity signal representing cardiac activity of the subject;

from a second portion of the images of the sequence of images corresponding to the body part area of the subject, extracting a motion signal representing motion of the body part area of the subject, the motion signal representing breathing and cardiac activity motion of the body part area of the subject;

determining a similarity measure between the cardiac activity signal and the motion signal over time;

comparing the similarity measure with a threshold; and

in response to the similarity measure meeting or exceeding the threshold, controlling an apnoea indicating device to indicate apnoea of the subject,

wherein when the subject is breathing normally, the breathing motion dominates the motion signal and when the subject undergoes apnoea, the motion signal is dominated by the cardiac activity motion, and

wherein when the cardiac activity motion dominates the motion signal, the similarity measure between the motion signal and the cardiac activity is high, and

wherein when the subject is breathing normally, the motion signal is dominated by the breathing of the subject and the similarity measure is low.

15. The monitor as claimed in claim 14, wherein when the motion signal is dominated by cardiac activity motion, the motion signal has a fundamental frequency at a heart rate of the subject, wherein when the subject is breathing normally, the motion signal has a fundamental frequency at the breathing rate and wherein the cardiac activity signal has a fundamental frequency at the heart rate of the subject, and wherein the one or more computer processors are further configured to:

comparing the fundamental frequencies of the motion signal and the cardiac activity signal.

16. The monitor according to claim 15, wherein the one or more computer processors are further configured to:

determine at least one of a Euclidean distance between the fundamental frequencies, a ratio between the fundamental frequencies, and an energy ratio of the fundamental frequencies.

17. The monitor according to claim 15, wherein the one or more computer processors are further configured to:

determine the similarity between the fundamental frequencies of the motion signal and the cardiac activity signal at a frequency of a heart rate of the subject.