Automated extraction of behavioral profile features

• US 10,542,021 B1
• Filed: 06/20/2016
• Issued: 01/21/2020
• Est. Priority Date: 06/20/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.), UnitedHealth Group Incorporated
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Mehr, Nima Sharifi
• Primary Examiner(s): Doan, Trang T
• Application Number:
  US15/187,532
• Time in Patent Office: 1,310 Days
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

Actions in an electronic environment are monitored during a learning period and behavior profiles generated using feature values for those actions. Subsequent behavior can be compared against the profiles to track the anomalies, or mismatches between features of incoming events and features of the profiles. A high percentage of mismatch can make a feature a candidate for exclusion from the behavioral profile. Normalization methods can be applied on features flagged as exclusion candidates. If any normalization sufficiently decreases the mismatch rate, the feature will not be excluded from the behavior profile. Any exclusion candidate feature which does not have an adequate mismatch value after normalization can be removed from tracked features of the corresponding profile. The behavior profile can be used to detect anomalous behavior that deviates from values of the behavior profile.

Claims

1. A computer-implemented method, comprising:

analyzing a plurality of actions detected in an electronic environment over an initial learning period, the plurality of actions being of at least one specified type;

generating a behavior profile using values determined for features representative of the plurality of actions;

determining, for at least a subset of the features, a respective mismatch value indicating a ratio of unexpected values to expected values for at least one feature of the subset of the features detected over a second learning period;

determining a first feature and a second feature where the respective mismatch value falls outside a range of acceptable mismatch values;

applying at least one first normalization method to the first feature, the at least one first normalization method causing additional information for one or more related types of data to be included in determining a new first mismatch value;

determining that the new first mismatch value falls within the range of acceptable mismatch values;

causing the first feature to remain included in the behavior profile;

applying at least one second normalization method to the second feature, the at least one second normalization method causing additional information for one or more related types of data to be included in determining a new second mismatch value;

determining that new second mismatch value for the second feature falls outside the range of acceptable mismatch values over a predetermined period of time, the second mismatch value monitored for convergence toward the range of acceptable mismatch values;

removing the second feature from the behavior profile, based at least in part on the new second mismatch value falling outside of the range of acceptable mismatch values;

detecting a subsequent action, corresponding to the at least one specified type, in the electronic environment, the subsequent action indicative of potentially anomalous behavior;

comparing values for the features of the subsequent action against the features of the behavior profile;

determining that a detected value for at least one feature for the subsequent action deviates from an expected value of a corresponding feature in the behavior profile by more than an acceptable amount; and

generating an alarm indicating potentially anomalous behavior in the electronic environment.

2. The computer-implemented method of claim 1, further comprising:

causing the first normalization method, corresponding to the new first mismatch value, to be associated with the first field in the behavior profile.

3. The computer-implemented method of claim 1, further comprising:

logging information for the subsequent action; and

using the information to update the expected values of the behavior profile.

4. The computer-implemented method of claim 1, further comprising:

monitoring the new first mismatch value during the second learning period at least until the first new mismatch value stabilizes to within a specified amount of variation.

5. The computer-implemented method of claim 1, further comprising:

determining a structure of a file associated with at least one action of the plurality of actions; and

flattening the file to determine the features representative of the at least one specified type.

6. A computer-implemented method, comprising:

processing a set of event data corresponding to one or more types of actions performed in an electronic environment;

generating, based at least in part upon the set of event data, a behavior profile indicating expected values for a set of features representative of a specific type of action of the one or more types of actions;

determining that a mismatch value for a determined feature, of the set of features, falls outside an acceptable mismatch value range;

applying a normalization method to the determined feature to determine a new mismatch value, the normalization method causing additional information for one or more related types of data to be included in determining the new mismatch value;

determining whether the new mismatch value falls outside the acceptable mismatch value range over a predetermined period of time, the new mismatch value monitored for convergence toward the acceptable mismatch value range;

removing the feature from the behavior profile, based at least in part on determining the new mismatch value falls outside the acceptable mismatch value range; and

providing the behavior profile to an analysis engine configured to detect anomalous behavior in subsequent actions of the specific type detected in the electronic environment.

7. The computer-implemented method of claim 6, further comprising:

causing the feature to remain included in the behavior profile if the new mismatch value falls within the acceptable mismatch value range.

8. The computer-implemented method of claim 6, further comprising:

determining the normalization method from a set of normalization methods based at least in part upon at least one of a type of the feature or a type of value expected for the feature.

9. The computer-implemented method of claim 6, further comprising:

receiving identification of the normalization method from a customer of the electronic environment who is associated with the behavior profile.

10. The computer-implemented method of claim 6, further comprising:

analyzing a plurality of actions detected in the electronic environment over an initial learning period, the actions being of at least the specific type; and

generating the behavior profile using values determined for features representative of the actions.

11. The computer-implemented method of claim 6, further comprising:

logging information for the subsequent actions; and

using the information to update the expected values of the behavior profile.

12. The computer-implemented method of claim 6, further comprising:

monitoring the new mismatch value during a second learning period at least until the mismatch value at least stabilizes to within a specified amount of variation.

13. The computer-implemented method of claim 6, further comprising:

determining a structure of a file associated with the specific type of action; and

flattening the file to determine the features representative of the specific type of action.

14. The computer-implemented method of claim 6, further comprising:

detecting a subsequent action, corresponding to the specific type, in the electronic environment;

comparing values for the features of the subsequent action against the features of the behavior profile;

determining that a detected value for at least one feature for the subsequent action deviates from an expected value of a corresponding feature in the behavior profile by more than an acceptable amount; and

performing a determined action.

15. The computer-implemented method of claim 14, wherein the determined action includes at least one of generating a notification, logging information, generating an alarm, blocking access, or modifying an access credential.

16. The computer-implemented method of claim 6, further comprising:

receiving, from a customer of the electronic environment, indication of at least one feature to exclude from the behavior profile.

17. A system, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the system to;

process a set of event logs corresponding to a specific type of action performed in an electronic environment;

generate a behavior profile indicating expected values for a set of features representative of the specific type of action, based at least in part upon the set of event logs;

determine that a mismatch value for a determined feature, of the set of features, falls outside an acceptable mismatch value range over a predetermined period of time, the new mismatch value monitored for convergence toward the acceptable mismatch value range;

apply a normalization method to the determined feature to determine a new mismatch value, the normalization method causing additional information for one or more related types of data to be included in determining the new mismatch value;

determine whether the new mismatch value falls outside the acceptable mismatch value range;

remove the feature from the behavior profile, based at least in part on determining the new mismatch value falls outside the acceptable mismatch value range; and

provide the behavior profile to an analysis engine configured to detect anomalous behavior in subsequent actions of the specific type detected in the electronic environment.

18. The system of claim 17, wherein the instructions when executed further cause the system to:

cause the feature to remain included in the behavior profile if the new mismatch value falls within the acceptable mismatch value range.

19. The system of claim 17, wherein the instructions when executed further cause the system to:

analyze a plurality of actions detected in the electronic environment over an initial learning period, the actions being of at least the specific type; and

generate the behavior profile using values determined for features representative of the actions.

20. The system of claim 17, wherein the instructions when executed further cause the system to:

detect a subsequent action, corresponding to the specific type, in the electronic environment;

compare values for the features of the subsequent action against the features of the behavior profile;

determine that a detected value for at least one feature for the subsequent action deviates from an expected value of a corresponding feature in the behavior profile by more than an acceptable amount; and

perform a determined action, wherein the determined action includes at least one of generating a notification, logging information, generating an alarm, blocking access, or modifying an access credential.