Systems and methods for configuring baby monitor cameras to provide uniform data sets for analysis and to provide an advantageous view point of babies

• US 10,645,349 B2
• Filed: 11/15/2018
• Issued: 05/05/2020
• Est. Priority Date: 04/08/2014
• Status: Active Grant

• Current Assignee: Udisense Inc.
• Original Assignee: Udisense Inc.
• Inventors: Glazer, Assaf
• Primary Examiner(s): Habib, Irfan
• Application Number:
  US16/191,712
• Publication Number:
  US 20190306465A1
• Time in Patent Office: 537 Days
• CPC Class Codes: A61B 2503/04 Babies, e.g. for SIDS detection
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/6891 Furniture
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Systems and methods for monitoring babies with cameras using a centralized computation and storage center that allows using visual output signals for computer vision and machine learning analysis and high-level reasoning of baby movements. The system comprises a camera located at a predefined working point above a baby'"'"'s crib, and one or more communication networks between components of the system including a web-based network for in-depth computer vision and machine learning analysis of the visual output signals by an analysis server.

Claims

1. A method for monitoring children while in cribs or on mattresses, the method performed in a system including a plurality of child monitor cameras configured to operate from a predefined working point with respect to the cribs or mattresses to thereby capture substantially uniform fields of view of the cribs or mattresses, the method comprising:

receiving image sequences of different children from the plurality of child monitor cameras;

storing the received image sequences;

processing the stored image sequences using one or more machine learning methods that classify regions of interest within the image sequences, recognize objects, features of the different children, and activities of the different children from the image sequences, and extract features from the stored image sequences for the different children;

performing analytical processing on the objects, features, or activities of the children from the image sequences received from at least different child monitor cameras to generate comparative data;

for any given image sequence of a given child received from a given child monitor camera, identifying events of interest by comparing recognized objects, features, or activities from the given image sequence from the given child monitor camera and one or more extracted features of the given child to the comparative data; and

transmitting data regarding the identified events of interest over a network to a first client device associated with the given child monitor camera.

2. The method of claim 1, further comprising providing the one or more machine learning methods with positive and negative information for classifying the regions of interest within the image sequences, recognizing the objects, features of the children, and activities of the children.

3. The method of claim 1, further comprising automatically adjusting a property of the given child monitor camera based on the one or more machine learning methods determining a crib area in a field of view of the given child monitor camera.

4. The method of claim 1, further comprises receiving sensory data from the plurality of child monitor cameras and including the sensory data in performing analytical processing and using the processed image sequences and the comparative data to identify events of interest.

5. The method of claim 1, wherein processing the stored image sequences further comprises extracting one or more of the following features from the stored image sequences for the children:

whether the given child is in or out of the crib or mattress, whether the given child is asleep or awake, whether a caretaker approached the crib or mattress, actions taken by a caretaker, measurements of the given child, and movements of the given child.

6. The method of claim 5, wherein the comparative data includes statistics on the extracted features of the given child.

7. The method of claim 6, wherein the statistics include one or more of:

sleep patterns over time, correlations between time child is put to sleep and hours in which child slept, significant movements of the child per night, summaries of coughing events, and number of interventions by caretakers per night.

8. The method of claim 5, wherein performing analytical processing comprises identifying common patterns from the image sequences received from at least different child monitor cameras.

9. The method of claim 8, wherein identifying the common patterns comprises identifying one or more of the following:

number of child movements, child measurements, child sleep patterns, and distances of child movements.

10. The method of claim 1, wherein identifying events of interest further comprises identifying one or more of:

the given child'"'"'s head is covered, the given child is awake, when the given child is crying, when the given child will wake up, the given child rolled over, the given child is climbing, the given child spits up, foreign object in the crib or mattress, aspiration event, flip event, seizures event, leg stuck event, and that the given child is sleeping on back or stomach.

11. A system for monitoring children while in cribs or on mattresses, the system comprising:

an analysis server connected to a network to receive image sequences of different children from a plurality of child monitor cameras that are configured to operate from a predefined working point with respect to the cribs or mattresses to thereby capture substantially uniform fields of view of the cribs or mattresses, the analysis server comprising;

a comparative data database storing the received image sequences;

an algorithmic logic module configured to process the image sequences using one or more machine learning methods to;

classify regions of interest within the image sequences;

recognize objects, features of the different children, and activities of or performed on the different children from the image sequences;

extract features from the stored image sequences for the different children;

perform analytical processing on the objects, features, or activities of or performed on the different children from the image sequences received from at least different child monitor cameras to generate comparative data; and

identify events of interest by comparing recognized objects, features, or activities from any given image sequence of a given child from a given child monitor camera and one or more extracted features of the given child to the comparative data; and

a services module for transmitting data regarding the identified events of interest over the network to client devices associated with the child monitor cameras.

12. The system of claim 11, wherein the algorithmic logic module is further configured to provide the one or more machine learning methods with positive and negative information for classifying the regions of interest within the image sequences, recognizing the objects, features of the children, and activities of the children.

13. The system of claim 11, wherein the child monitor cameras include one or more sensors to measure sensory data, the one or more sensors for measuring one or more of the following:

temperature, humidity, noise, and light intensity.

14. The system of claim 13, wherein the analysis server is configured to receive the measured sensory data over the network, and wherein the algorithmic logic module is further configured to include the measured sensory data in its analytical processing and identification of events of interest.

15. The system of claim 11, wherein the algorithmic logic module is further configured to generate statistics on the recognized features from the image sequences of a given child, and wherein the services module is configured to transmit a report on the statistics to a first client device associated with the given child monitor camera.

16. The system of claim 15, wherein the statistics include one or more of:

sleep patterns over time, correlations between time child is put to sleep and hours in which child slept, significant movements of the child per night, summaries of coughing events, and number of interventions by caretakers per night.

17. The system of claim 11, wherein the algorithmic logic module is further configured to compare recognized features from image sequences from at least different child monitor cameras and identify common patterns between the recognized features.

18. The system of claim 17, wherein identifying the common patterns comprises the algorithmic logic module identifying one or more of the following:

number of child movements, child measurements, child sleep patterns, and distances of child movements.

19. The system of claim 11, wherein the algorithmic logic module is further configured to extract one or more of the following features from the stored image sequences for the children:

whether the given child is in or out of the crib or mattress, whether the given child is asleep or awake, whether a caretaker approached the crib or mattress, actions taken by a caretaker, measurements of the given child, and movements of the given child.

20. The system of claim 11, wherein the algorithmic logic module is further configured to identify one or more of:

the given child'"'"'s head is covered, the given child is awake, when the given child is crying, when the given child will wake up, the given child rolled over, the given child is climbing, the given child spits up, foreign object in the crib or mattress, aspiration event, flip event, seizures event, leg stuck event, and that the given child is sleeping on back or stomach.