REMOTE BIOMETRIC MONITORING SYSTEM

• US 20160364617A1
• Filed: 06/14/2016
• Published: 12/15/2016
• Est. Priority Date: 06/15/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Knit Health, Inc.
• Inventors: JANSSENS, David Carl, SHAPIRO, Evan David, CARLUCCI, Adam Daniel, SILBERSCHATZ, Paul Joseph
• Granted Patent: US 10,169,662 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2503/04 Babies, e.g. for SIDS detection
• CPC Class Codes: A61B 2560/0242 adapted to measure environmental factors, e.g. temperature, pollution
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/1113 Local tracking of patients, e.g. in a hospital or private home
• CPC Class Codes: A61B 5/4806 Sleep evaluation A61B5/4821 takes precedence; devices for inducing sleep A61M21/02
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Remote biometric monitoring systems may include a digital camera having a digital sensor, a processor, and a memory, all enclosed in a common housing. The processor of the camera may locally execute one or more algorithms to perform computer vision analysis of captured images of a sleeping subject, thereby determining an activity state of the subject. The activity state may include a sleep state. One or more environmental control devices may be adjusted automatically by the system based at least in part on the activity state.

Claims

1. A system for remotely monitoring a sleeping subject, the system comprising:

a digital camera configured to capture images of a subject, the camera including a digital image sensor, one or more processors in communication with the digital image sensor, and a memory, wherein the digital image sensor, the one or more processors, and the memory are enclosed in a same housing;

an environmental sensor in communication with the one or more processors of the digital camera, the environmental sensor configured to detect a physical characteristic of a local environment of the subject;

an environmental control device in communication with the one or more processors of the digital camera, the environmental control device configured to produce a change in the local environment of the subject; and

a set of instructions stored in the memory of the digital camera and executable locally by the one or more processors to;

receive a sensed input from the environmental sensor and a plurality of time-sequenced images of the subject from the digital image sensor;

determine, using an artificial intelligence module, a position of a body region of the subject in a subset of images of the time-sequenced plurality of images;

identify, using the position of the body region determined by the artificial intelligence module to perform a biometric analysis on the plurality of time-sequenced images, an activity state of the subject; and

cause a change in the environmental control device based on the sensed input and the activity state of the subject.

2. The system of claim 1, wherein the plurality of time-sequenced images are uncompressed.

3. The system of claim 1, wherein the biometric analysis includes determining whether the subject is moving by performing an actigraphic analysis of the plurality of time-sequenced images of the subject.

4. The system of claim 1, wherein identifying the activity state of the subject includes identifying a sleep state of the subject.

5. The system of claim 1, wherein the subset of images consists of a single image.

6. The system of claim 1, wherein the biometric analysis further includes determining a respiration pattern of the subject.

7. The system of claim 6, wherein causing the change in the environmental control device is further based on the respiration pattern of the subject.

8. The system of claim 1, further comprising a tripod support having the digital camera articulatably attached thereto, wherein the tripod includes a pair of short legs each having a first permanent length, and a long leg having a second permanent length at least fifty percent longer than the first permanent length.

9. The system of claim 1, wherein the artificial intelligence module comprises a neural network.

10. The system of claim 1, further comprising a remote notification device in communication with the digital camera, the remote notification device configured to provide information relating to the activity state of the subject to a user located out of sight of the subject.

11. A system for remotely monitoring a sleeping person, the system comprising:

a digital camera configured to capture images of a person, the camera including a digital image sensor, one or more processors in communication with the digital image sensor, and a memory, wherein the digital image sensor, the one or more processors, and the memory are enclosed in a same housing;

a remote notification device in communication with a user located out of sight of the person; and

a set of instructions stored in the memory of the digital camera and executable locally by the one or more processors to;

receive a plurality of time-sequenced images of the person from the digital image sensor;

determine, using an artificial intelligence module, a position of a body region of the person in at least one image of the time-sequenced plurality of images;

identify, using the position of the body region determined by the artificial intelligence to perform a biometric analysis on the plurality of time-sequenced images, an activity state of the person; and

communicate information corresponding to the activity state of the person to the user via the remote notification device.

12. The system of claim 11, wherein the biometric analysis includes determining a movement state of the person by performing an actigraphic analysis of the plurality of time-sequenced images.

13. The system of claim 11, wherein the activity state of the person includes a sleep state.

14. The system of claim 11, wherein the set of instructions are further executable locally by the one or more processors to determine a respiration pattern of the person.

15. A method for remotely monitoring a sleep state of a human subject, the method comprising:

capturing a plurality of time-sequenced images of a human subject using a digital camera having a digital image sensor, one or more processors in communication with the digital image sensor, and a memory, wherein the digital image sensor, the one or more processors, and the memory are enclosed in a same housing;

determining, using the one or more processors to execute a machine intelligence module in the memory of the digital camera, a position of a body region of the subject in at least one image of the time-sequenced plurality of images;

identifying, using the one or more processors to execute a biometric analysis on the body region of the plurality of time-sequenced images, an activity state of the subject; and

causing a change in a local environment of the subject based at least in part on the activity state of the subject.

16. The method of claim 15, wherein the activity state of the subject includes a sleep state of the subject.

17. The method of claim 15, wherein executing the biometric analysis includes determining whether the subject is moving by executing an actigraphic analysis of the plurality of time-sequenced images.

18. The method of claim 15, further including determining a respiration pattern of the subject.

19. The method of claim 18, the respiration pattern comprising a respective duration of each inhalation and each exhalation.

20. The method of claim 18, further comprising:

displaying a changing light pattern on a remote notification device, the light pattern corresponding to the respiration pattern of the subject.