Sleep monitoring system and method

• US 10,058,272 B2
• Filed: 12/04/2014
• Issued: 08/28/2018
• Est. Priority Date: 12/13/2013
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Heinrich, Adrienne, Van Vugt, Henriette Christine, Jeanne, Vincent
• Primary Examiner(s): D ABREU, MICHAEL JOSEPH
• Application Number:
  US15/101,433
• Publication Number:
  US 20160310046A1
• Time in Patent Office: 1,363 Days
• CPC Class Codes: A61B 2503/04 Babies, e.g. for SIDS detection
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/113 occurring during breathing
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/4806 Sleep evaluation A61B5/4821 takes precedence; devices for inducing sleep A61M21/02
• CPC Class Codes: A61B 5/4818 Sleep apnoea
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B

Abstract

The sleep monitoring system uses a motion vector estimator to determine motion vectors as a function of location in camera images. A signal processing system with an input coupled to the motion vector estimator computes a measure of turning motion, by summing motion vectors from respective locations within an image. Pose changes are detected based on the measure of turning motion. The measure of turning motion may be computed from a sum of components of the motion vectors that are normal to the major axis of a body area in the image, which is determined based on image content changes in a set of images from the camera. To avoid false turning detection due to leg kicking or similar motion the detected turning motion may be reduced to zero if it is not detected in a sufficiently large part of the body area.

Claims

1. A monitoring system for monitoring a person on a bed, the system comprising

a camera for obtaining a set of images of a person on the bed;

a motion vector estimator with an input coupled to the camera, configured to determine motion vectors as a function of location in each successive image of the set of images obtained during a time interval;

a signal processing system with an input coupled to the motion vector estimator;

wherein the signal processing system is configured to

determine, within each successive image, (i) a map of image locations for blocks in each respective image associated with a body area of the person on the bed and (ii) a direction of a major axis of the body area on the bed from the determined map of image locations, wherein the direction is determined by a size of the body area on the bed along that direction of the major axis being larger than a size of the body area on the bed in all other directions, or at least not smaller, and wherein the direction of the major axis of the body area on the bed is adaptable in response to deviations of the major axis from a longitudinal direction of the bed;

compute a measure of turning motion of the person on the bed, wherein the computation of the measure of turning motion of the person on the bed comprises (i) using (i)(a) the direction of the major axis of the body area on the bed and (i)(b) components of the determined motion vectors of each block in each respective image that are normal to the direction of the major axis of the body area on the bed and (ii) subsequently summing these components over the blocks from the respective map of image locations associated with the body area of the person on the bed within each successive image of the set of images; and

detect pose state changes of the person on the bed based on the computed measure of turning motion of the person.

2. The monitoring system according to claim 1, wherein the signal processing system is further configured to

detect time intervals wherein the measure of turning motion indicates less than a predetermined amount of motion;

and

assign different pose states to successive ones of the detected time intervals, each based on signs of the measure of turning motion in further time intervals surrounding the time interval, and/or on a position of the time interval in a sequence of time intervals in which the detected time intervals occur.

3. The monitoring system according to claim 2, wherein the pose states include an on the belly pose state, an on the back pose state and an on the side pose state, and wherein the signal processing system is configured to use accumulated sizes of the measure of turning motion between the time intervals and/or time duration between the time intervals to select between using direct transitions between the on the belly pose state and the on the back pose state and transitions from the on the belly pose state or the on the back pose state to the on the side pose state.

4. The monitoring system according to claim 3, wherein the signal processing system is further configured to use a sign of the measure of turning motion between the time intervals to select between using transitions to an on the left side pose state and an on the right side pose state.

5. The monitoring system according to claim 2, wherein the signal processing system is further configured to generate an alert signal (i) when a predetermined pose state has been assigned and/or (ii) when a same pose state remains assigned for more than a predetermined time interval.

6. The monitoring system according to claim 5, wherein the signal processing system is further configured to perform breathing detection based on images from the camera and to generate the alert signal on condition that (i) no breathing with parameters within a predetermined range has been detected and/or (ii) a change of detected breathing parameters has been detected, while the predetermined pose state has been assigned.

7. The monitoring system according to claim 2, wherein the signal processing system is further configured to perform face detection and/or breathing detection based on images from the camera and to set part of the assigned pose states based on results of face detection and/or breathing detection.

8. The monitoring system according to claim 1, wherein the signal processing system is further configured to

determine, within each successive image, the image locations associated with the body area based on image content changes in the set of images from the camera, captured during a time window including a time point of capturing each respective image or adjacent to said time point.

9. The monitoring system according to claim 8, wherein the signal processing system is further configured to select a region of interest in images from the camera for face detection and or heart beat detection dependent on the direction of the major axis.

10. The monitoring system according to claim 1, wherein the signal processing system is further configured to detect whether motion vectors in at least a predetermined fraction of the image locations in the body area or a part thereof exceed a threshold size and, if not, then to decrease the measure of turning motion.

11. The monitoring system according to claim 1, wherein the signal processing system is further configured to detect whether motion vectors in at least a first predetermined fraction of the image locations in the body area exceed a threshold size, and motion vectors in at least a second predetermined fraction of the image locations in a first end part of the body area along the major axis exceed the threshold size and, if not both, then to decrease the measure of turning motion.

12. The monitoring system according to claim 1, wherein the signal processing system is further configured to determine the measure of turning motion by accumulating sums of motion vectors from respective locations as a function of time and low pass filtering the accumulated sums.