Systems and methods for analyzing emotional responses to online interactions

• US 10,419,375 B1
• Filed: 06/14/2016
• Issued: 09/17/2019
• Est. Priority Date: 06/14/2016
• Status: Active Grant

• Current Assignee: Gen Digital Inc.
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Sokolov, Ilya, Newstadt, Keith
• Primary Examiner(s): Huq, Farzana B
• Application Number:
  US15/181,457
• Time in Patent Office: 1,190 Days
• Field of Search: 709204, 709206
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

The disclosed computer-implemented method for analyzing emotional responses to online interactions may include (1) identifying an online interaction of a user, (2) detecting an emotional response of the user to the online interaction by monitoring one or more emotional indicators of the user during the online interaction and determining, based on an evaluation of the one or more emotional indicators, that the emotional response of the user is outside an expected range, and (3) performing a security action in response to determining that the user'"'"'s emotional response is outside the expected range. Various other methods, systems, and computer-readable media are also disclosed.

Claims

1. A computer-implemented method for analyzing emotional responses to online interactions, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:

identifying an online interaction between a user and a second user, the online interaction comprising an electronic message received by the user;

detecting an emotional response of the user to the online interaction by;

monitoring one or more emotional indicators of the user during the online interaction; and

determining, based on an evaluation of the one or more emotional indicators, that the emotional response of the user to the online interaction;

comprises a negative emotional response of the user to the online interaction; and

is outside an expected range;

evaluating the one or more emotional indicators to assign a category to an emotional relationship between the user and the second user; and

performing a security action to protect the user based on the category assigned to the emotional relationship between the user and the second user and in response to determining that the emotional response of the user to the online interaction;

comprises a negative emotional response of the user to the online interaction; and

is outside the expected range;

wherein the emotional response of the user to the online interaction further comprises a change in a measurement of at least one emotional indicator of the user that occurs in response to the online interaction.

2. The computer-implemented method of claim 1, wherein determining that the emotional response of the user is outside the expected range comprises:

identifying previous measurements for the one or more emotional indicators of the user to determine the expected range of the one or more emotional indicators for the user; and

determining that the emotional response of the user is outside the expected range for the user.

3. The computer-implemented method of claim 1, further comprising:

analyzing measurements of the one or more emotional indicators for a group of users to establish the expected range.

4. The computer-implemented method of claim 1, wherein:

monitoring the one or more emotional indicators of the user during the online interaction comprises accumulating, for a plurality of users, emotional indicator measurements pertaining to the online interaction;

determining, based on the evaluation of the one or more emotional indicators, that the emotional response of the user is outside the expected range comprises determining, based on the evaluation of the emotional indicator measurements, that for at least a subset of the plurality of users the online interaction elicits emotional responses outside the expected range; and

the security action is performed in response to the determination that the online interaction elicits emotional responses outside the expected range for at least the subset of the plurality of users.

5. The computer-implemented method of claim 1, wherein:

the method further comprises;

accumulating, for a plurality of users, emotional indicator measurements pertaining to the online interaction; and

determining, based on the evaluation of the emotional indicator measurements, that the online interaction elicits emotional responses outside the expected range for the user and within the expected range for a subset of the plurality of users; and

the security action is performed in response to the determination that the online interaction elicits emotional responses outside the expected range for the user and within the expected range for the subset of the plurality of users.

6. The computer-implemented method of claim 1, wherein:

the method further comprises;

identifying a plurality of online interactions between the user and the second user;

monitoring one or more emotional indicators of the user during the plurality of online interactions; and

evaluating the one or more emotional indicators to assign the category to the emotional relationship between the user and the second user comprises evaluating the one or more emotional indicators of the user during the plurality of online interactions.

7. The computer-implemented method of claim 1, wherein the electronic message comprises at least one of:

an email message;

a text message;

social network content;

an audio message; and

a video message.

8. The computer-implemented method of claim 1, wherein the security action comprises at least one of:

blocking a message from a sender;

blocking a social network message;

reporting the online interaction to an administrator of an online service;

reporting the online interaction to a parent of the user;

reporting the online interaction to an account administrator for the user; and

reporting the online interaction to a school administrator for the user.

20. The method of claim 1, wherein the security action comprises monitoring additional emotional indicators for the user.

9. A system for analyzing emotional responses to online interactions, the system comprising:

an identification module, stored in memory, that identifies an online interaction of a user, the online interaction comprising an electronic message received by the user;

an analysis module, stored in memory, that;

detects an emotional response of the user to the online interaction by;

monitoring one or more emotional indicators of the user during the online interaction; and

determining, based on an evaluation of the one or more emotional indicators, that the emotional response of the user to the online interaction;

comprises a negative emotional response of the user to the online interaction; and

is outside an expected range; and

evaluates the one or more emotional indicators to assign a category to an emotional relationship between the user and the second user;

a security module, stored in memory, that performs a security action to protect the user based on the category assigned to the emotional relationship between the user and the second user and in response to determining that the emotional response of the user to the online interaction;

comprises a negative emotional response of the user to the online interaction; and

is outside the expected range; and

at least one physical processor configured to execute the identification module, the analysis module, and the security module;

wherein the emotional response of the user to the online interaction further comprises a change in a measurement of at least one emotional indicator of the user that occurs in response to the online interaction.

10. The system of claim 9, wherein the analysis module determines that the emotional response of the user is outside the expected range by:

identifying previous measurements for the one or more emotional indicators of the user to determine the expected range of the one or more emotional indicators for the user; and

determining that the emotional response of the user is outside the expected range for the user.

11. The system of claim 9, wherein the analysis module analyzes measurements of the one or more emotional indicators for a group of users to establish the expected range.

12. The system of claim 9, wherein:

the analysis module;

monitors the one or more emotional indicators of the user during the online interaction by accumulating, for a plurality of users, emotional indicator measurements pertaining to the online interaction; and

determines, based on the evaluation of the one or more emotional indicators, that the emotional response of the user is outside the expected range by determining, based on the evaluation of the emotional indicator measurements, that for at least a subset of the plurality of users the online interaction elicits emotional responses outside the expected range; and

the security module performs the security action in response to the determination that the online interaction elicits emotional responses outside the expected range for at least the subset of the plurality of users.

13. The system of claim 9, wherein:

the analysis module;

accumulates, for a plurality of users, emotional indicator measurements pertaining to the online interaction; and

determines, based on the evaluation of the emotional indicator measurements, that the online interaction elicits emotional responses outside the expected range for the user and within the expected range for a subset of the plurality of users; and

the security module performs the security action in response to the determination that the online interaction elicits emotional responses outside the expected range for the user and within the expected range for the subset of the plurality of users.

14. The system of claim 9, wherein:

the identification module identifies a plurality of online interactions between the user and the second user;

the analysis module;

monitors one or more emotional indicators of the user during the plurality of online interactions; and

evaluates the one or more emotional indicators to assign the category to the emotional relationship between the user and the second user by evaluating the one or more emotional indicators of the user during the plurality of online interactions.

15. The system of claim 9, wherein the electronic message comprises at least one of:

an email message;

a text message;

social network content;

an audio message; and

a video message.

16. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:

identify an online interaction between a user and a second user, the online interaction comprising an electronic message received by the user;

detect an emotional response of the user to the online interaction by;

monitoring one or more emotional indicators of the user during the online interaction; and

determining, based on an evaluation of the one or more emotional indicators, that the emotional response of the user to the online interaction;

comprises a negative emotional response of the user to the online interaction; and

is outside an expected range;

evaluate the one or more emotional indicators to assign a category to an emotional relationship between the user and the second user; and

perform a security action to protect the user based on the category assigned to the emotional relationship between the user and the second user and in response to determining that the emotional response of the user to the online interaction;

comprises a negative emotional response of the user to the online interaction; and

is outside the expected range;

wherein the emotional response of the user to the online interaction further comprises a change in a measurement of at least one emotional indicator of the user that occurs in response to the online interaction.

17. The non-transitory computer-readable medium of claim 16, wherein the one or more computer-readable instructions cause the computing device to determine that the emotional response of the user is outside the expected range by:

identifying previous measurements for the one or more emotional indicators of the user to determine the expected range of the one or more emotional indicators for the user; and

determining that the emotional response of the user is outside the expected range for the user.

18. The non-transitory computer-readable medium of claim 16, wherein the one or more computer-readable instructions cause the computing device to analyze measurements of the one or more emotional indicators for a group of users to establish the expected range.

19. The non-transitory computer-readable medium of claim 16, wherein the one or more computer-readable instructions cause the computing device to:

monitor the one or more emotional indicators of the user during the online interaction by accumulating, for a plurality of users, emotional indicator measurements pertaining to the online interaction;

determine, based on the evaluation of the one or more emotional indicators, that the emotional response of the user is outside the expected range by determining, based on the evaluation of the emotional indicator measurements, that for at least a subset of the plurality of users the online interaction elicits emotional responses outside the expected range; and

perform the security action in response to the determination that the online interaction elicits emotional responses outside the expected range for at least the subset of the plurality of users.