Trusted user identification and management for home automation systems

• US 10,049,515 B2
• Filed: 08/24/2016
• Issued: 08/14/2018
• Est. Priority Date: 08/24/2016
• Status: Active Grant

• Current Assignee: EchoStar Technologies International Corporation (Echostar Corporation)
• Original Assignee: EchoStar Technologies International Corporation (Echostar Corporation)
• Inventors: Greene, Gregory
• Primary Examiner(s): Kholdebarin, Iman K
• Application Number:
  US15/246,079
• Publication Number:
  US 20180061158A1
• Time in Patent Office: 720 Days
• Field of Search: 382118
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/50 Maintenance of biometric data or enrolment thereof
• CPC Class Codes: G07C 9/00904 for hotels, motels, office buildings or the like
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition

Abstract

Various arrangements for determining and setting an authorization level for an unknown person are presented. Facial recognition may be performed on a received video stream. An authorized user may be recognized in the video feed. Additionally, an unknown person may be identified in the video feed. A provisional authorization level may be granted for the unknown person based on proximity between the unknown person and the authorized user in the received video stream.

Claims

1. A home automation system comprising:

a video camera that transmits a video stream to a home automation host system via wireless communication; and

the home automation host system, comprising;

a wireless communication interface;

one or more processors; and

a memory communicatively coupled with and readable by the one or more processors and having stored therein processor-readable instructions which, when executed by the one or more processors, cause the one or more processors to;

store a database of users, wherein each user is linked with stored visual identification data used to identify the user in received video and an authorization level;

receive the video stream from the video camera;

perform facial recognition on a plurality of persons that appear in the video stream;

identify, in the plurality of persons that appear in the video stream, an authorized user based on stored visual identification from the stored database of users;

identify, in the plurality of persons that appear in the video stream, an unknown person;

create a record for the unknown person in the database of users based on identifying the unknown person;

determine proximity between the unknown person and the authorized user based on the received video stream;

grant the unknown person a provisional authorization level based on the determined proximity between the unknown person and the authorized user in the received video stream; and

suppress transmission of a notification indicating the presence of the unknown person based on the unknown person having been granted at least the provisional authorized status based on the proximity between the unknown person and the authorized user in the received video stream.

2. The home automation system of claim 1, wherein the proximity between the unknown person and the authorized user is a temporal proximity and wherein the processor-readable instructions, when executed, further cause the one or more processors to:

determine that the temporal proximity between the unknown person and the authorized user in the video stream is less than a threshold period of time.

3. The home automation system of claim 1, wherein the proximity between the unknown person and the authorized user is a physical proximity and wherein the processor-readable instructions, when executed, further cause the one or more processors to:

determine, based on the video stream, that the authorized user and the unknown person appear in a field of view of the video camera at a same time.

4. The home automation system of claim 1, wherein the processor-readable instructions, when executed, further cause the one or more processors to:

capture an image of the unknown person from the received video stream; and

store the image of the unknown person as part of the created record for the unknown person in the database of users.

5. The home automation system of claim 4, wherein the processor-readable instructions, when executed, further cause the one or more processors to:

output, for presentation to the authorized user, via a user interface, information, comprising the image, from the record for the unknown person;

output a request that requests the authorized user input an updated authorization level for the unknown person;

receive, via the user interface, a response to the request, the response comprising an indication of the updated authorization level for the unknown person; and

modify the created record for the unknown person to indicate the updated authorization level provided in the response to the request.

6. The home automation system of claim 1, wherein the processor-readable instructions, when executed, further cause the one or more processors to:

cease recording of the video stream based on the unknown person having been granted at least the provisional authorized status based on the proximity between the unknown person and the authorized user.

7. The home automation system of claim 1, wherein the processor-readable instructions, when executed, further cause the one or more processors to:

access a contact list of a first mobile device, the first mobile device being linked with the authorized user in the database of users, wherein the contact list comprises a plurality of identifiers linked with a plurality of persons;

receive an identifier from a second mobile device, the second mobile device being linked with the unknown person;

determine the identifier received from the second mobile device matches an identifier of the plurality of identifiers of the accessed contact list; and

update the record for the unknown person to include a name for the unknown person obtained from the contact list, the name for the unknown person being linked in the contact list with the identifier of the plurality of identifiers of the accessed contact list.

8. The home automation system of claim 1, wherein the processor-readable instructions, when executed, further cause the one or more processors to:

revoking the provisional authorization level of the unknown person based on the unknown person not being detected within proximity of any authorized user in the received video stream for at least a threshold duration of time and the unknown person not having been granted any permanent authorized status.

9. A method for determining and setting an authorization level for an unknown person, the method comprising:

storing, by a home automation host system, a database of users, wherein each user is linked with stored visual identification data used to identify the user in received video and the authorization level;

receiving, by the home automation host system, a video stream from a video camera;

performing, by the home automation host system, facial recognition on a plurality of persons that appear in the video stream;

identifying, by the home automation host system, in the plurality of persons that appear in the video stream, an authorized user based on stored visual identification from the stored database of users;

identifying, by the home automation host system, in the plurality of persons that appear in the video stream, the unknown person;

creating, by the home automation host system, a record for the unknown person in the database of users based on identifying the unknown person;

determine, by the home automation host system, proximity between the unknown person and the authorized user based on the received video stream;

setting, by the home automation host system, a provisional authorization level for the unknown person based on the determined proximity between the unknown person and the authorized user in the received video stream, wherein the provisional authorization level alters a distribution of notifications related to the unknown person;

capturing, by the home automation host system, an image of the unknown person from the received video stream;

storing, by the home automation host system, the image of the unknown person as part of the created record for the unknown person in the database of users;

outputting, by the home automation host system, to a mobile device for presentation to the authorized user, the created record for the unknown person, wherein the image of the unknown person from the record is output for presentation to the authorized user;

outputting, by the home automation host system, a request that requests the authorized user input an updated authorization level for the unknown person;

receiving, by the home automation host system, a response to the request, the response comprising an indication of an updated authorization level for the unknown person; and

modifying, by the home automation host system, the created record for the unknown person to indicate the updated authorization level provided in the response to the request.

10. The method for determining and setting the authorization level for the unknown person of claim 9, wherein the proximity between the unknown person and the authorized user is a temporal proximity and the method further comprises:

determining that the temporal proximity between the unknown person and the authorized user in the video stream is less than a threshold period of time.

11. The method for determining and setting the authorization level for the unknown person of claim 9, wherein the proximity between the unknown person and the authorized user is a physical proximity and the method further comprises, based on the video stream, that the authorized user and the unknown person appear in a field of view of the video camera at a same time.

12. The method for determining and setting the authorization level for the unknown person of claim 9, further comprising:

suppressing, by the home automation host system, transmission of a notification indicating the presence of the unknown person based on the unknown person having been granted at least the provisional authorization level based on the proximity between the unknown person and the authorized user in the received video stream.

13. The method for determining and setting the authorization level for the unknown person of claim 9, further comprising:

ceasing, by the home automation host system, recording of the video stream based on the unknown person having been granted at least the provisional authorization level based on the proximity between the unknown person and the authorized user.

14. A non-transitory processor-readable medium, comprising processor-readable instructions that cause one or more processors to:

store a database of users, wherein each user is linked with stored visual identification data used to identify the user in received video and an authorization level;

receive a video stream from a video camera that operates as part of a home automation system;

record the video stream from the video camera;

perform facial recognition on a plurality of persons that appear in the video stream;

identify, in the plurality of persons that appear in the video stream, an authorized user based on stored visual identification from the stored database of users;

identify, in the plurality of persons that appear in the video stream, an unknown person;

create a record for the unknown person in the database of users based on identifying the unknown person;

determine proximity between the unknown person and the authorized user based on the received video stream; and

grant the unknown person a provisional authorization level based on the determined proximity between the unknown person and the authorized user in the received video stream; and

cease to record the video stream based on the unknown person having been granted at least the provisional authorization level based on the proximity between the unknown person and the authorized user.

15. The non-transitory processor-readable medium of claim 14, wherein the proximity between the unknown person and the authorized user is a temporal proximity and wherein the processor-readable instructions further cause the one or more processors to determine that the temporal proximity between the unknown person and the authorized user in the video stream is less than a threshold period of time.

16. The non-transitory processor-readable medium of claim 14, wherein the proximity between the unknown person and the authorized user is a physical proximity and wherein the processor-readable instructions further cause the one or more processors to determine, based on the video stream, that the authorized user and the unknown person appear in a field of view of the video camera at a same time.

17. The non-transitory processor-readable medium of claim 14, wherein the processor-readable instructions further cause the one or more processors to:

output, for presentation to the authorized user, the created record for the unknown person, wherein an image of the unknown person is output for presentation to the authorized user;

output a request that requests the authorized user input an updated authorization level for the unknown person;

receive, via a user interface, a response to the request, the response comprising an indication of the updated authorization level for the unknown person; and

modify the created record for the unknown person to indicate the updated authorization level provided in the response to the request.