Systems and methods for configuring baby monitor cameras to provide uniform data sets for analysis and to provide an advantageous view point of babies

• US 10,165,230 B2
• Filed: 11/28/2016
• Issued: 12/25/2018
• Est. Priority Date: 04/08/2014
• Status: Active Grant

• Current Assignee: Udisense Inc.
• Original Assignee: Udisense Inc.
• Inventors: Glazer, Assaf
• Primary Examiner(s): Patel, Jayanti K
• Assistant Examiner(s): Habib, Irfan
• Application Number:
  US15/361,702
• Publication Number:
  US 20170078620A1
• Time in Patent Office: 757 Days
• CPC Class Codes: A61B 2503/04 Babies, e.g. for SIDS detection
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/6891 Furniture
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Systems and methods for monitoring babies with cameras using a centralized computation and storage center that allows using visual output signals for computer vision and machine learning analysis and high-level reasoning of baby movements. The system comprises a camera located at a predefined working point above a baby'"'"'s crib, and one or more communication networks between components of the system including a web-based network for in-depth computer vision and machine learning analysis of the visual output signals by an analysis server.

Claims

1. A method for monitoring children while in cribs or on mattresses, the method performed in a system including a plurality of child monitor cameras positioned and configured to capture substantially uniform fields of view of the cribs or mattresses, the method comprising:

receiving image sequences of different children from the plurality of child monitor cameras;

storing the received image sequences;

processing the stored image sequences using one or more machine learning methods to classify regions of interest within the image sequences and to recognize objects, features of the children, and activities of the children from the image sequences based on the regions of interest and to extract features from the stored image sequences for the children;

performing analytical processing on the processed image sequences by comparing the recognized objects, features, or activities of the different children from the image sequences received from the plurality of child monitor cameras and identifying patterns based on the recognized objects, features, or activities associated with the different children;

for any given image sequences received from a given child monitor camera, using the processed image sequences and analytical processing to identify events of interest by comparing recognized objects, features, or activities from the given child monitor camera and one or more extracted features for the given child to the identified patterns and identifying the events of interest based on the comparison of the recognized objects, features, or activities from the given child monitor camera and extracted features having similarities to or differences from a given one of the identified patterns; and

transmitting data regarding the identified events of interest over a network to a first client device associated with the given child monitor camera.

2. The method of claim 1, wherein the child monitor cameras are designed to be positioned at substantially uniform heights above the cribs or mattresses to capture the substantially uniform fields of view.

3. The method of claim 2, wherein transmitting data comprises transmitting data to automatically adjust a property of the given child monitor camera.

4. The method of claim 1, wherein the child monitor cameras comprise one or more sensors to measure sensory data, and wherein the method further comprises receiving the sensory data from the plurality of child monitor cameras and including the sensory data in executing the steps of performing analytical processing and using the processed image sequences and analytical processing to identify events of interest.

5. The method of claim 1, wherein processing the stored image sequences comprises extracting one or more of the following features from the image sequences for each child:

whether child is in or out of the crib or mattress, whether child is asleep or awake, whether a caretaker approached the crib or mattress, actions taken by a caretaker, measurements of the child, and movements of the child.

6. The method of claim 5, the method further comprising generating statistics on the extracted features of the given child and transmitting a report on such statistics to the first client device associated with the given child monitor camera.

7. The method of claim 6, wherein transmitting the report comprises transmitting the report regarding one or more of the following statistics:

sleep patterns over time, correlations between time child is put to sleep and hours in which child slept, significant movements of the child per night, summaries of coughing events, and number of interventions by caretakers per night.

8. The method of claim 5, wherein performing analytical processing comprises comparing extracted features from image sequences from a plurality of different child cameras to identify common patterns between features.

9. The method of claim 8, wherein identifying the common patterns comprises identifying one or more of the following:

number of child movements, child measurements, child sleep patterns, and distances of child movements.

10. The method of claim 1, wherein using the processed image sequences and analytical processing to identify events of interest comprises identifying one or more of the following events of interest regarding the given child:

a notification that the child'"'"'s head is covered with a blanket, a notification when the child is awake, a notification when the child is crying, a notification when the child will wake up, and a notification that the child is sleeping on his back or stomach.

11. A system for monitoring children while in cribs or on mattresses, the system comprising:

a plurality of child monitor cameras configured to capture substantially uniform fields of view of the cribs or mattresses and to transmit image sequences of different children from the plurality of child monitor cameras over a network; and

an analysis server connected to the network to receive the transmitted image sequences of the different children from the plurality of child monitor cameras, the analysis server comprising;

a comparative data database storing the received image sequences;

an algorithmic logic module configured to process the image sequences using one or more machine learning methods to;

classify regions of interest within the image sequences;

recognize objects, features of the children, and activities of or performed on the children from the image sequences based on the regions of interest and to extract features from the stored image sequences for the children;

perform analytical processing on the processed image sequences by comparing the recognized objects, features, or activities of or performed on the different children from the image sequences received from the plurality of child monitor cameras and identifying patterns based on the recognized objects, features, or activities associated with the different children; and

use the processed image sequences and analytical processing to identify events of interest by comparing recognized objects, features, or activities from a given child monitor camera and one or more extracted features for the given child to the identified patterns and identifying the events of interest based on the comparison of the recognized objects, features, or activities from the given child monitor camera and extracted features having similarities to or differences from a given one of the identified patterns; and

a services module for transmitting data regarding the identified events of interest over the network to client devices associated with the child monitor cameras.

12. The system of claim 11, wherein the child monitor cameras comprise means for positioning the cameras at substantially uniform heights above the cribs or mattresses to capture the substantially uniform fields of view.

13. The system for claim 12, wherein the means for positioning comprises a mount including one or more arms and an attachment.

14. The system of claim 11, wherein the child monitor cameras each comprise one or more sensors to measure sensory data, the one or more sensors for measuring one or more of the following:

temperature, humidity, noise, and light intensity.

15. The system of claim 14, wherein the child monitor cameras are further configured to transmit the measured sensory data to the analysis server over the network, and wherein the algorithmic logic module is further configured to include the measured sensory data in its analytical processing and identification of events of interest.

16. The system of claim 11, wherein the child monitor cameras each comprise an infrared light emitting diode light which illuminates at a wavelength of approximately 850 nm to 940 nm such that a glare is not visible to a child being monitored by the camera.

17. The system of claim 11, wherein the child monitor cameras each comprise an adjustable rotational lens mount.

18. The system of claim 17, wherein the child monitor cameras are configured to adjust the rotational lens mount in response to receiving data transmitted by the analysis server.

19. The system of claim 11, wherein the algorithmic logic module is further configured to generate statistics on the recognized features from the image sequences of a given child, and wherein the services module is configured to transmit a report on such statistics to a first client device associated with the given child monitor camera.

20. The system of claim 11, wherein the algorithmic logic module is configured to compare recognized features from image sequences from a plurality of different child cameras to identify common patterns between the recognized features.

21. The system of claim 11, wherein the algorithmic logic module is configured to process the image sequences to recognize from the image sequences activities of caretakers performed on the children.