Screen-analysis based device security

• US 10,803,408 B2
• Filed: 09/17/2018
• Issued: 10/13/2020
• Est. Priority Date: 06/15/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Sharifi, Matthew, Wang, Kai, Petrou, David
• Primary Examiner(s): Doan, Trang T
• Application Number:
  US16/133,395
• Publication Number:
  US 20190019110A1
• Time in Patent Office: 757 Days
• CPC Class Codes: G06F 21/316 by observing the pattern of computer usage, e.g. typical user behaviour
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 63/083 using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04W 12/06 Authentication
• CPC Class Codes: H04W 12/10 Integrity
• CPC Class Codes: H04W 12/126 Anti-theft arrangements, e.g. protection against subscriber identity module [SIM] cloning
• CPC Class Codes: H04W 12/30 Security of mobile devices; Security of mobile applications
• CPC Class Codes: H04W 12/63 Location-dependent; Proximity-dependent
• CPC Class Codes: H04W 12/68 Gesture-dependent or behaviour-dependent
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

Systems and methods are provided for a content-based security for computing devices. An example method includes identifying content rendered by a mobile application, the content being rendered during a session, generating feature vectors from the content and determining that the feature vectors do not match a classification model. The method also includes providing, in response to the determination that the feature vectors do not match the classification model, a challenge configured to authenticate a user of the mobile device. Another example method includes determining a computing device is located at a trusted location, capturing information from a session, the information coming from content rendered by a mobile application during the session, generating feature vectors for the session, and repeating this until a training criteria is met. The method also includes training a classification model using the feature vectors and authenticating a user of the device using the trained classification model.

Claims

1. An electronic device comprising:

at least one processor;

memory storing instructions that, when executed by the at least one processor, cause the electronic device to;

when a first user profile is active for the electronic device;

identify, based on one or more screen capture events performed at the electronic device, content rendered by the electronic device;

generate feature vectors from the content rendered by the electronic device;

process the feature vectors using a classification model stored locally at the electronic device, the classification model trained to predict whether activity at the electronic device corresponds to the first user profile or to one or more additional user profiles, including a second user profile that differs from the first user profile;

generate, based on processing the feature vectors using the classification model, a first confidence score for the first user profile and a second confidence score for the second user profile;

determine, based on the first confidence score and the second confidence score, that the first user profile is no longer active, and the second user profile is now active for the electronic device; and

initiate a profile switch, to the second user profile, at the electronic device responsive to determining that that the first user profile is no longer active, and the second user profile is now active for the electronic device.

2. The electronic device of claim 1, wherein in initiating the profile switch, to the second user profile, the electronic device is to initiate the profile switch automatically, resulting in use, by the electronic device, of second user profile data for the second user profile.

3. The electronic device of claim 2, wherein in initiating the profile switch automatically, the electronic device is to automatically sign out the first user profile and automatically sign on the second user profile.

4. The electronic device of claim 1, wherein in generating the feature vectors the electronic device is to generate the feature vectors from the content rendered by the electronic device and from metadata that is in addition to the content rendered by the electronic device.

5. The electronic device of claim 4, wherein the metadata that is in addition to the content rendered by the electronic device includes one or multiple of:

an application that rendered the content, applications active at the electronic device when the content was rendered, and a location of the electronic device.

6. The electronic device of claim 1, wherein in initiating the profile switch, to the second user profile, the electronic device is to present an interface to authenticate the second user profile.

7. The electronic device of claim 6, wherein the interface to authenticate the second user profile includes a challenge using prior content captured from a prior screen capture event that occurred at the electronic device when the second user profile was authenticated.

8. The electronic device of claim 1, wherein the instructions, when executed by the at least one processor, further cause the electronic device to:

generate first positive training examples for the first user profile based on first past screen capture events, wherein generating the first positive training examples for the first user profile based on the first past screen capture events is based on the first past screen capture events occurring in a trusted environment for the first user profile; and

train the classification model using the first positive training examples for the first user profile.

9. The electronic device of claim 8, wherein the instructions, when executed by the at least one processor, further cause the electronic device to:

generate negative training examples for the second user profile based on the first past screen capture events, wherein generating the negative training examples for the second user profile based on the first past screen capture events is based on the first past screen capture events occurring in a trusted environment for the first user profile; and

train the classification model using the negative training examples for the second user profile.

10. The electronic device of claim 9, wherein the instructions, when executed by the at least one processor, further cause the electronic device to:

generate second positive training examples for the second user profile based on second past screen capture events, wherein generating the second positive training examples for the second user profile based on the second past screen capture events is based on the second past screen capture events occurring in a trusted environment for the second user profile; and

train the classification model using the second positive training examples for the second user profile.

11. A method performed by one or more processors when a first user profile is active for an electronic device, the method comprising:

identifying, based on one or more screen capture events performed at the electronic device, content rendered by the electronic device;

generating feature vectors from the content rendered by the electronic device;

processing the feature vectors using a classification model trained to predict whether activity at the electronic device corresponds to the first user profile or to one or more additional user profiles, including a second user profile that differs from the first user profile;

generating, based on processing the feature vectors using the classification model, a first confidence score for the first user profile and a second confidence score for the second user profile;

determining, based on the first confidence score and the second confidence score, that the first user profile is no longer active, and the second user profile is now active for the electronic device; and

initiating a profile switch, to the second user profile, at the electronic device responsive to determining that that the first user profile is no longer active, and the second user profile is now active for the electronic device.

12. The method of claim 11, wherein initiating the profile switch, to the second user profile, comprises initiating the profile switch automatically, resulting in use, by the electronic device, of second user profile data for the second user profile.

13. The method of claim 12, wherein initiating the profile switch automatically comprises automatically signing out the first user profile and automatically signing on the second user profile.

14. The method of claim 11, wherein generating the feature vectors comprises generating the feature vectors from the content rendered by the electronic device and from metadata that is in addition to the content rendered by the electronic device.

15. The method of claim 14, wherein the metadata that is in addition to the content rendered by the electronic device includes one or multiple of:

an application that rendered the content, applications active at the electronic device when the content was rendered, and a location of the electronic device.

16. The method of claim 11, wherein initiating the profile switch, to the second user profile, comprise presenting an interface to authenticate the second user profile.

17. The method of claim 16, wherein presenting the interface to authenticate the second user profile comprises presenting a challenge using prior content captured from a prior screen capture event that occurred at the electronic device when the second user profile was authenticated.

18. The method of claim 11, further comprising:

generating first positive training examples for the first user profile based on first past screen capture events, wherein generating the first positive training examples for the first user profile based on the first past screen capture events is based on the first past screen capture events occurring in a trusted environment for the first user profile; and

training the classification model using the first positive training examples for the first user profile.

19. The method of claim 18, further comprising:

generating negative training examples for the second user profile based on the first past screen capture events, wherein generating the negative training examples for the second user profile based on the first past screen capture events is based on the first past screen capture events occurring in a trusted environment for the first user profile; and

training the classification model using the negative training examples for the second user profile.

20. The method of claim 19, further comprising:

generating second positive training examples for the second user profile based on second past screen capture events, wherein generating the second positive training examples for the second user profile based on the second past screen capture events is based on the second past screen capture events occurring in a trusted environment for the second user profile; and

training the classification model using the second positive training examples for the second user profile.