Method and device for monitoring a baby for interaction

• US 10,694,956 B2
• Filed: 07/21/2015
• Issued: 06/30/2020
• Est. Priority Date: 07/21/2014
• Status: Active Grant

• Current Assignee: Withings S.A.
• Original Assignee: Withings S.A.
• Inventors: Dumoulin, Amaury, Mathias, Guillaume, Carreel, Eric
• Primary Examiner(s): Jackson, Gary
• Assistant Examiner(s): Kuo, Jonathan T
• Application Number:
  US14/804,873
• Publication Number:
  US 20160015277A1
• Time in Patent Office: 1,806 Days
• CPC Class Codes: A61B 2503/04 Babies, e.g. for SIDS detection
• CPC Class Codes: A61B 2560/0242 adapted to measure environmental factors, e.g. temperature, pollution
• CPC Class Codes: A61B 5/0004 characterised by the type of physiological signal transmitted
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: A61B 5/7207 of noise induced by motion artifacts
• CPC Class Codes: G01N 33/0047 for organic compounds
• CPC Class Codes: G06F 2218/10 by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30076 Plethysmography
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 21/0208 Combination with audio or video communication, e.g. combination with "baby phone" function
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control

Abstract

A method for video evaluation of the heart rate and/or respiratory rate of a baby, in particular under dim or nocturnal conditions, in a device comprising a video camera and a source of infrared light, the method comprising the steps of:

A1—illuminating the baby with the infrared source,

A2—capturing video images of the baby,

B1—determining the position of the baby'"'"'s head by finding an ellipse forming an outline of the head,

B2—identifying an area of interest centered on a selected ellipse,

C—evaluating, by plethysmographic analysis of the area of interest, the heart rate and/or respiratory rate of the baby,

and associated system.

Claims

1. A method for video evaluation of the heart rate and/or respiratory rate of a baby in dim or nocturnal conditions, in a device comprising a video camera having a wide viewing angle of at least 90°

and a source of infrared light, the method comprising the steps of;

A1—

illuminating the baby with the infrared source,

A2—

capturing video images of the baby,

B1—

determining the position of the baby'"'"'s head by detecting edge segments forming separate curved portions of an ellipse coinciding with the outline of the head,

B2—

identifying an area of interest centered on the ellipse,

C—

evaluating, by photoplethysmographic analysis of the area of interest, the heart rate and/or respiratory rate of the baby,

D—

identifying a macroscopic movement of the baby, in which case step C—

of photoplethysmographic analysis is suspended,

D2—

identifying a return to a stable state where there is no macroscopic movement by the baby, in which case steps B1—

and B2—

are executed once again, step C—

of photoplethysmographic analysis being resumed with the new area of interest determined in step B2—

.

2. The method according to claim 1, wherein the images are captured in color and the photoplethysmographic analysis is based on the red component of the video images.

3. The method according to claim 1, wherein at step B1—

, a method of image convolution with statistical thresholding is used to find edges or edge segments in the image.

4. The method according to claim 1, wherein at step B1—

, candidate ellipses are formed from the identified edge segments, and a selection filter is applied to the candidate ellipses to retain only one selected ellipse coinciding with a likely outline of the baby'"'"'s head.

5. The method according to claim 1, wherein in step C—

, a digital filter with passband of 0.75 Hz-3.5 Hz is applied to the light signal spatially averaged over the area of interest, to obtain a filtered signal.

6. The method according to claim 5, wherein a validity criterion concerning the relevance of the filtered signal is defined, for selectively validating or not validating the heart rate information, the validity criterion being based on the signal-to-noise ratio of the filtered signal.

7. The method according to claim 1, wherein the captured images are each divided into an array of sub-images of identical size.

8. The method according to claim 7, wherein at step B1—

the sub-images having a neutral content, meaning with no variation in color or intensity within the area of the sub-image, are eliminated, and after step B2—

the sub-images located outside said ellipse are eliminated.

9. A device for video evaluation of the heart rate and/or respiratory rate of a baby in dim or nocturnal conditions, comprising a video camera having a wide viewing angle of at least 90°

, a source of infrared light, and a computing unit, wherein the infrared source is configured to illuminate the baby, the video camera is configured to capture video images of the baby, and the computing unit is configured to;

A1—

illuminating the baby with the infrared source,

A2—

capturing video images of the baby,

B1—

determining the position of the baby'"'"'s head by detecting edge segments forming separate curved portions of an ellipse substantially coinciding with the outline of the head,

B2—

identifying an area of interest centered on the ellipse,

C—

evaluating, by photoplethysmographic analysis of the area of interest, the heart rate and/or respiratory rate of the baby,

D—

identifying a macroscopic movement of the baby, in which case step C—

of photoplethysmographic analysis is suspended, and

D2—

identifying a return to a stable state where there is no macroscopic movement by the baby, in which case steps B1-B2 are executed once again, step C—

of photoplethysmographic analysis being resumed with the new area of interest determined in step B2—

.

10. The device according to claim 9, comprising a wireless communication interface for transmitting the heart rate and/or respiratory rate data of the baby to a remote entity.