Thoughtful elderly monitoring in a smart home environment

• US 10,467,879 B2
• Filed: 10/19/2017
• Issued: 11/05/2019
• Est. Priority Date: 10/19/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Stefanski, Mark, Matsuoka, Yoky
• Primary Examiner(s): Lim, Steven
• Assistant Examiner(s): Ma, Kam Wan
• Application Number:
  US15/788,354
• Publication Number:
  US 20190122522A1
• Time in Patent Office: 747 Days
• Field of Search: 3405731
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 17/10 Actuation by presence of smoke or gases , e.g. automatic alarm devices for analysing flowing fluid materials by the use of optical means
• CPC Class Codes: G08B 21/0423 detecting deviation from an expected pattern of behaviour or schedule
• CPC Class Codes: G08B 25/08 using communication transmission lines G08B13/19658, G08B21/0286, G08B25/016 take precedence

Abstract

Various arrangements are presented for monitoring a resident of a residence. A confidence assessment may be performed based on a plurality of smart home devices in the residence. The residence may be identified as eligible for monitoring of the resident based on the confidence assessment. A learning process may be performed to create an ordinary behavior model. Data that is received from the plurality of smart home devices may be monitored to identify data indicative of behavior considered unusual based on the ordinary behavior model. An alert may be created that identifies the behavior and identifies how the behavior contrasts with the ordinary behavior model.

Claims

1. A method for monitoring a resident, the method comprising:

performing a confidence assessment based on a plurality of smart home devices being present within a residence linked with a user account, wherein the resident resides at the residence and performing the confidence assessment comprises;

calculating a confidence metric based on;

a number of the plurality of smart home devices that are eligible to participate in monitoring the resident;

a second number of the plurality of smart home devices that are eligible to participate in the monitoring and are power constrained; and

responses to a questionnaire received from an administrator device; and

comparing the calculated confidence metric to a confidence metric threshold;

determining whether the residence is eligible for monitoring of the resident based on the confidence assessment;

performing a learning process over a period of time during which resident activity data is collected from the plurality of smart home devices and analyzed to create an ordinary behavior model;

following determining that the residence is eligible for monitoring of the resident and the learning process being performed, providing a notification that monitoring is active;

monitoring data received from the plurality of smart home devices to identify data indicative of behavior considered unusual based on the ordinary behavior model;

creating an alert that identifies the behavior and identifies how the behavior contrasts with the ordinary behavior model; and

sending, to the administrator device linked with the user account, the alert that identifies the behavior and identifies how the behavior contrasts with the ordinary behavior model.

2. The method for monitoring the resident of claim 1, wherein performing the confidence assessment comprises identifying the number of the plurality of smart home devices that are eligible to participate in the monitoring.

3. The method for monitoring the resident of claim 2, wherein performing the confidence assessment further comprises identifying the second number of the plurality of smart home devices that are eligible to participate in the monitoring and are powered-constrained devices.

4. The method for monitoring the resident of claim 3, further comprising:

in response to the monitoring being activated, activating a process at each exclusively battery-powered smart home device of the plurality of smart home devices that defines one or more rules indicative of when data indicative of a behavior of the resident should be stored for periodic scheduled transmission to a monitoring server system or the data indicative of the behavior of the resident should be transmitted immediately to the monitoring server system.

5. The method for monitoring the resident of claim 3, wherein performing the confidence assessment further comprises providing the questionnaire to the administrator device linked with the user account, wherein the questionnaire requires that a user of the administrator device identify:

a specific location of each smart home device of the plurality of smart home devices within the residence; and

a plurality of types of worrisome scenarios of which the user of the administrator device desires to be notified.

6. The method for monitoring the resident of claim 5, wherein performing the confidence assessment further comprises providing the questionnaire to the administrator device linked with the user account, wherein the questionnaire requires that the user of the administrator device provide:

an indication of a number of residents that live in the residence; and

an indication that no cats or dogs live with the resident.

7. The method for monitoring the resident of claim 1, wherein the plurality of smart home devices are selected from the group consisting of:

a smart home smoke detector;

a smart home carbon monoxide detector;

a smart indoor security camera;

a smart outdoor security camera;

a smart thermostat;

a smart home assistant device;

a smart security system;

a smart window/door sensor;

a smartphone; and

a smart doorbell device.

8. The method for monitoring the resident of claim 1, wherein each of the plurality of smart home devices includes either a video camera, microphone, or a motion sensor.

9. The method for monitoring the resident of claim 1, further comprising:

outputting, via a smart home device of the plurality of smart home devices, the alert.

10. A method for monitoring a resident, the method comprising:

performing a confidence assessment based on a plurality of smart home devices being present within a residence linked with a user account, wherein the resident resides at the residence and the confidence assessment comprises;

monitoring the resident using the plurality of smart home devices over a trial period of time to generate a body of trial monitoring data, the trial period of time being sufficient to encompass at least a plurality of daily- and/or day-of-week-specific activity routines of the resident;

processing the trial monitoring data to determine whether the plurality of smart home devices are sufficient in type, number, and location to sufficiently track the resident through their daily- and/or day-of-week-specific activity routines according to a predetermined threshold criterion;

determining that the plurality of smart home devices are not sufficient in at least one of type, number, and location to sufficiently track the resident according to the predetermined threshold criterion; and

responsive to said determining, sending to an administrator device a notification that the residence is not eligible for said monitoring of the resident.

11. The method for monitoring the resident of claim 10, wherein said predetermined threshold criterion includes that, for at least a threshold percentage of each day of the trial period, a location of the resident within the residence is identifiable solely using the trial monitoring data.

12. The method of claim 11, wherein said threshold percentage is at least 95 percent.

13. The method of claim 11, wherein said predetermined threshold criterion includes that, for at least the threshold percentage of each day of the trial period, a breathing rate or heartbeat rate of the resident within the residence is identifiable solely using the trial monitoring data.

14. The method of claim 10, wherein said trial period of time is a default value of at least seven days, and wherein said performing the confidence assessment is carried out without requiring user input.

15. The method of claim 10, wherein said trial period of time is received from a user via the administrative device.

16. A system for monitoring a resident, the system comprising:

a plurality of smart home devices installed within a residence linked with a user account;

an application executed by an administrator device; and

a cloud-based host system comprising;

one or more processors; and

a memory communicatively coupled with and readable by the one or more processors and having stored therein processor-readable instructions which, when executed by the one or more processors, cause the one or more processors to;

perform a confidence assessment based on the plurality of smart home devices being present within the residence linked with the user account, wherein the resident resides at the residence and performing the confidence assessment comprises;

calculating a confidence metric based on;

a number of the plurality of smart home devices that are eligible to participate in monitoring the resident, and responses to a questionnaire received from the application

comparing the calculated confidence metric to a confidence metric threshold;

determine whether the residence is eligible for monitoring of the resident based on the confidence assessment;

perform a learning process over a period of time during which resident activity data is collected from the plurality of smart home devices and analyzed to create an ordinary behavior model;

following, determining that the residence is eligible for monitoring of the resident and the learning process being performed, provide a notification that monitoring is active;

monitor data received from the plurality of smart home devices to identify data indicative of behavior considered unusual based on the ordinary behavior model;

create an alert that identifies the behavior and identifies how the behavior contrasts with the ordinary behavior model; and

send, to the administrator device linked with the user account, the alert that identifies the behavior and identifies how the behavior contrasts with the ordinary behavior model.

17. The system for monitoring the resident of claim 16, wherein the processor-readable instructions that cause the one or more processors to perform the confidence assessment further comprises processor-readable instructions that cause the one or more processors to identify the number of the plurality of smart home devices that are eligible to participate in the monitoring.

18. The system for monitoring the resident of claim 17, wherein the processor-readable instructions that cause the one or more processors to perform the confidence assessment further comprises processor-readable instructions that cause the one or more processors to identify a second number of the plurality of smart home devices that are eligible to participate in the monitoring and are powered-constrained devices.

19. The system for monitoring the resident of claim 18, wherein the processor-readable instructions are further configured to cause the one or more processors to:

in response to the monitoring being activated, activate a process at each power-constrained smart home device of the plurality of smart home devices that defines one or more rules indicative of when data indicative of a behavior of the resident should be stored for periodic scheduled transmission to an monitoring server system or the data indicative of the behavior of the resident should be transmitted immediately to the monitoring server system.

20. The system for monitoring the resident of claim 18, wherein the processor-readable instructions that cause the one or more processors to perform the confidence assessment further comprises processor-readable instructions that cause the one or more processors to provide the questionnaire to the administrator device linked with the user account, wherein the questionnaire requires that a user of the administrator device identify:

a specific location of each smart home device of the plurality of smart home devices within the residence; and

a plurality of types of worrisome scenarios of which the administrator user desires to be notified.

21. The system for monitoring the resident of claim 20, wherein the processor-readable instructions that cause the one or more processors to perform the confidence assessment further comprises processor-readable instructions that cause the one or more processors to provide the questionnaire to the administrator device linked with the user account, wherein the questionnaire requires that the user of the administrator device provide:

an indication of a number of residents that live in the residence; and

an indication that no cats or dogs live with the resident.

22. The system for monitoring the resident of claim 16, wherein the plurality of smart home devices are selected from the group consisting of:

a smart home smoke detector;

a smart home carbon monoxide detector;

a smart indoor security camera;

a smart outdoor security camera;

a smart thermostat;

a smart home assistant device;

a smart security system;

a smart window/door sensor;

a smartphone; and

a smart doorbell device.

23. The system for monitoring the resident of claim 16, wherein a smart home device of the plurality of smart home devices, outputs the alert.

24. A non-transitory processor-readable medium comprising processor-readable instructions configured to cause one or more processors to:

perform a confidence assessment based on a plurality of smart home devices being present within a residence linked with a user account, wherein a resident resides at the residence and performing the confidence assessment comprises;

calculating a confidence metric based on;

a number of the plurality of smart home devices that are eligible to participate in monitoring the resident;

a second number of the plurality of smart home devices that are eligible to participate in the monitoring and are power constrained; and

responses to a questionnaire received from an administrator device; and

comparing the calculated confidence metric to a confidence metric threshold;

determine Whether the residence is eligible for monitoring of the resident based on the confidence assessment;

perform a learning process over a period of time during, which resident activity data is collected from the plurality of smart home devices and analyzed to create an ordinary behavior model;

following determining that the residence is eligible for monitoring of the resident and the learning process being performed, provide a notification that monitoring is active;

monitor data received from the plurality of smart home devices to identify data indicative of behavior considered unusual based on the ordinary behavior model;

create an alert that identifies the behavior and identifies how the behavior contrasts with the ordinary behavior model; and

cause to be sent, to the administrator device linked with the user account, the alert that identifies the behavior and identifies how the behavior contrasts with the ordinary behavior model.