BABY MONITOR

• US 20180053393A1
• Filed: 07/24/2017
• Published: 02/22/2018
• Est. Priority Date: 08/19/2016
• Status: Active Grant

• Current Assignee: Innovative Health Monitoring LLC
• Original Assignee: EGW Technologies LLC
• Inventors: White, Eric Gregory, Abrams, David Robert
• Granted Patent: US 10,825,314 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2503/04 Babies, e.g. for SIDS detection
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/725 using specific filters therefor, e.g. Kalman or adaptive filters A61B5/316 take precedence
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7278 Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 7/00 Instruments for auscultation
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G08B 21/0208 Combination with audio or video communication, e.g. combination with "baby phone" function
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 23/634 Warning indications
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Systems and methods are described herein for monitoring and capturing information associated with target subjects, such as babies. For example, the systems and methods provide a capture device, such as a camera, that captures capture time of flight (TOF) data from a target subject. Using the TOF data, the systems and methods, via a display device associated with (e.g., paired to) the capture device, present information about the target subject, such as information indicating movement of the target subject, information indicating a breathing rate of the target subject, information indicating a heart rate of the target subject, and so on.

Claims

1. A method, comprising:

receiving time of flight (TOF) data measured from a target subject;

parsing the TOF data to extract a phase frame and an amplitude frame from the TOF data;

determining pixel velocities for pixels from distance data of the phase frame;

down-selecting high confidence pixels using a mask generated from confidence data of the amplitude frame;

filtering the high confidence pixels;

thresholding the filtered high confidence pixels;

for each frame, performing blob analysis to the frame to identify blob centroids within the frame;

determining spatial co-location between each frame by clustering identified blob centroids across a time series of frames;

determining statistics for the clustered blob centroids;

aggregating the identified centroids into classes of clusters based on an analysis of the determined statistics for the clustered blob centroids;

extracting one or more features associated with the classes of clusters; and

determining the extracted one or more features represent movement or breathing of the target subject.

2. The method of claim 1, wherein the received TOF data is captured by a TOF sensor proximate to the target subject.

3. The method of claim 1, wherein parsing the TOF data to extract a phase frame and an amplitude frame from the TOF data includes extracting distance data from the phase frame and confidence data from the amplitude frame.

4. The method of claim 1, wherein determining pixel velocities for pixels from distance data of the phase frame includes subtracting distance data extracted from a previous phase frame from distance data extracted from the phase frame.

5. The method of claim 1, wherein down-selecting high confidence pixels using a mask generated from confidence data of the amplitude frame includes:

identifying pixels that do not satisfy a threshold value for an associated amplitude; and

nullifying the identified pixels using the mask generated from the confidence data.

6. The method of claim 1, wherein filtering the high confidence pixels includes:

applying a Gaussian low-pass filter to the high confidence pixels to spatially smooth the high confidence pixels; and

performing temporal smoothing to select temporally correlated pixels of the high confidence pixels.

7. The method of claim 1, further comprising:

after thresholding the filtered high confidence pixels, performing morphological erosion to identify relatively larger areas of movement within the frame.

8. The method of claim 1, wherein performing blob analysis to the frame to identify blob centroids within the frame includes:

performing blob analysis to identify centroids of morphologically connected regions between frames; and

aggregating the identified centroids of morphologically connected regions across all frames.

9. The method of claim 1, wherein aggregating the identified centroids into classes of clusters based on an analysis of the determined statistics for the clustered blob centroids includes identifying the aggregated centroids using an iterated k-means algorithm.

10. The method of claim 1, wherein determining the extracted one or more features represent movement or breathing of the target subject includes:

processing the extracted one or more features using a fuzzy controller; and

classifying truth values output by the fuzzy controller using an artificial neural network to determine the extracted one or more features are associated with movement of the target subject or breathing of the target subject.

11. A non-transitory computer-readable medium whose contents, when executed by a computing system, cause the computing system to perform a method for determining a movement status of a target subject, the method comprising:

receiving time of flight (TOF) data associated with a target subject;

determining pixel velocities for pixels associated with one or more areas of the target subject; and

determining whether the one or more areas of the target subject are moving based on the determined pixel velocities.

12. The non-transitory computer-readable medium of claim 11, further comprising:

outputting a movement status for the target subject based on determining whether the one or more areas of the target subject are moving, wherein the movement status includes a status of movement of the target subject and a status of breathing of the target subject.

13. The non-transitory computer-readable medium of claim 11, wherein receiving time of flight (TOF) data associated with a target subject includes receiving 3D TOF data from a TOF sensor that is proximate to the target subject and captures the 3D TOF data from the target subject.

14. The non-transitory computer-readable medium of claim 11, wherein determining pixel velocities for pixels associated with one or more areas of the target subject includes determining, for each pixel in a depth image of the target subject, a change in distance over time between depth frames of the depth image.

15. The non-transitory computer-readable medium of claim 11, wherein determining pixel velocities for pixels associated with one or more areas of the target subject includes filtering depth frames of a depth image of the target subject to identify pixel velocities associated with movement of the target subject.

16. A baby monitor configured to display images of a monitored baby, the baby monitor comprising:

a movement determination system configured to;

access depth image data captured from images of the monitored baby;

determine pixel velocities for pixels associated with one or more areas of the monitored baby; and

determine whether the one or more areas of the monitored baby are moving based on the determined pixel velocities.

17. The baby monitor of claim 16, wherein the movement determination system is further configured to:

output a movement status for the monitored baby based on determining whether the one or more areas of the monitored baby are moving, wherein the movement status includes a status of movement of the monitored baby and a status of breathing of the monitored baby.

18. The baby monitor of claim 16, wherein the accessed depth image data associated with a monitored baby includes 3D time of flight (TOF) data captured by a TOF sensor of a capture device monitoring the baby and paired to a display device that displays the images of the monitored baby.

19. The baby monitor of claim 16, wherein the movement determination system determines pixel velocities by calculating, for each pixel in a depth image of the target subject, a change in distance over time between depth frames of the depth image.

20. The baby monitor of claim 16, wherein the baby monitor includes a capture device that captures the images of the monitored baby and contains the movement determination system.