Systems and methods for detecting and scoring anomalies

• US 9,800,601 B2
• Filed: 01/20/2017
• Issued: 10/24/2017
• Est. Priority Date: 09/05/2015
• Status: Active Grant

• Current Assignee: NuData Security Inc. (MasterCard Incorporated)
• Original Assignee: NuData Security Inc. (MasterCard Incorporated)
• Inventors: Bailey, Christopher Everett, Lukashuk, Randy, Richardson, Gary Wayne
• Primary Examiner(s): DOLLY, KENDALL LYNN
• Application Number:
  US15/411,805
• Publication Number:
  US 20170134414A1
• Time in Patent Office: 277 Days
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 21/316 by observing the pattern of computer usage, e.g. typical user behaviour
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/44 Program or device authentication
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 2221/033 Test or assess software
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: G06F 2221/2137 Time limited access, e.g. to a computer or data
• CPC Class Codes: G06F 2221/2145 Inheriting rights or properties, e.g., propagation of permissions or restrictions within a hierarchy
• CPC Class Codes: G06F 7/08 Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry by merging two or more sets of carriers in ordered sequence G06F7/16
• CPC Class Codes: H04L 2463/144 Detection or countermeasures against botnets
• CPC Class Codes: H04L 41/20 Network management software packages
• CPC Class Codes: H04L 43/12 Network monitoring probes
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 : Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 : Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1466 : Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/30 : Profiles
• CPC Class Codes: H04L 67/303 : Terminal profiles
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Systems and methods for detecting and scoring anomalies. In some embodiments, a method is provided, comprising acts of: (A) identifying a plurality of values of an attribute, each value of the plurality of values corresponding respectively to a digital interaction of the plurality of digital interactions; (B) dividing the plurality of values into a plurality of buckets; (C) for at least one bucket of the plurality of buckets, determining a count of values from the plurality of values that fall within the at least one bucket; (D) comparing the count of values from the plurality of values that fall within the at least one bucket against historical information regarding the attribute; and (E) determining whether the attribute is anomalous based at least in part on a result of the act (D).

Claims

1. A computer-implemented method for analyzing a plurality of first digital interactions observed from a first time period, the method comprising acts of:

(A) identifying a plurality of first values of an attribute, each value of the plurality of first values corresponding respectively to a digital interaction of the plurality of first digital interactions observed from the first time period;

(B) dividing the plurality of first values into a plurality of buckets;

(C) for at least one bucket of the plurality of buckets, determining a count of values from the plurality of first values that fall within the at least one bucket;

(D) comparing, by at least one processor, the count of values from the plurality of first values that fall within the at least one bucket against historical information regarding the attribute, wherein;

the historical information regarding the attribute comprises an expected count for the at least one bucket;

the count of values from the plurality of first values that fall within the at least one bucket is compared against the expected count for the at least one bucket;

the expected count for the at least one bucket comprises a count of values from a plurality of second values of the attribute that fall within the at least one bucket;

each value of the plurality of second values corresponds respectively to a digital interaction of a plurality of second digital interactions;

the plurality of second digital interactions were observed from a second time period, the second time period having a same length as the first time period; and

the first time period occurs after the second time period;

(E) determining whether the attribute is anomalous over the first time period, based at least in part on a result of the act (D); and

(F) in response to determining that the attribute is anomalous over the first time period, storing, in a profile, information regarding the attribute.

2. The method of claim 1, wherein:

each value of the plurality of first values comprises a time measurement between a first point and a second point in the corresponding digital interaction; and

each bucket of the plurality of buckets comprises a range of time measurements.

3. The method of claim 1, wherein:

the act (B) of dividing the plurality of first values into a plurality of buckets comprises applying a hash-modding operation to each value of the plurality of first values; and

each bucket of the plurality of buckets corresponds to a residue of the hash-modding operation.

4. The method of claim 3, further comprising acts of:

(G) recording a plurality of observations with respect to the attribute, each observation of the plurality of observations being recorded from a corresponding digital interaction of the plurality of first digital interactions; and

(H) deriving each value of the plurality of first values based on the observation recorded from the corresponding digital interaction.

5. The method of claim 1, wherein the act (E) comprises:

determining if the count of values from the plurality of first values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least a selected threshold amount, wherein the attribute is determined to be anomalous in response to determining that the count of values from the plurality of first values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least the selected threshold amount.

6. The method of claim 1, further comprising an act of:

(G) dividing the plurality of second values of the attribute into the plurality of buckets.

7. The method of claim 1, wherein the plurality of buckets comprises a plurality of first buckets, and wherein the method further comprises acts of:

(G) determining if the count of values from the plurality of first values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least a selected threshold amount; and

(H) in response to determining that the count of values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least the selected threshold amount, dividing the plurality of values into a plurality of second buckets, wherein there are more second buckets than first buckets.

8. The method of claim 1, further comprising an act of:

(G) selecting a plurality of attributes, the plurality of attributes comprising the attribute, wherein acts (A)-(F) are performed for each attribute of the plurality of attributes.

9. A system for analyzing a plurality of first digital interactions observed from a first time period, the system comprising:

at least one processor; and

at least one computer-readable storage device having stored thereon executable instructions which, when executed, program the at least one processor to perform a method comprising acts of;

(A) identifying a plurality of first values of an attribute, each value of the plurality of first values corresponding respectively to a digital interaction of the plurality of first digital interactions observed from the first time period;

(B) dividing the plurality of first values into a plurality of buckets;

(C) for at least one bucket of the plurality of buckets, determining a count of values from the plurality of first values that fall within the at least one bucket;

(D) comparing the count of values from the plurality of first values that fall within the at least one bucket against historical information regarding the attribute, wherein;

the historical information regarding the attribute comprises an expected count for the at least one bucket;

the count of values from the plurality of first values that fall within the at least one bucket is compared against the expected count for the at least one bucket;

the expected count for the at least one bucket comprises a count of values from a plurality of second values of the attribute that fall within the at least one bucket;

each value of the plurality of second values corresponds respectively to a digital interaction of a plurality of second digital interactions;

the plurality of second digital interactions were observed from a second time period, the second time period having a same length as the first time period; and

the first time period occurs after the second time period;

(E) determining whether the attribute is anomalous over the first time period, based at least in part on a result of the act (D); and

(F) in response to determining that the attribute is anomalous over the first time period, storing, in a profile, information regarding the attribute.

10. The system of claim 9, wherein:

each value of the plurality of first values comprises a time measurement between a first point and a second point in the corresponding digital interaction; and

each bucket of the plurality of buckets comprises a range of time measurements.

11. The system of claim 9, wherein:

the act (B) of dividing the plurality of first values into a plurality of buckets comprises applying a hash-modding operation to each value of the plurality of first values; and

each bucket of the plurality of buckets corresponds to a residue of the hash-modding operation.

12. The system of claim 11, wherein the method further comprises acts of:

(G) recording a plurality of observations with respect to the attribute, each observation of the plurality of observations being recorded from a corresponding digital interaction of the plurality of first digital interactions; and

(H) deriving each value of the plurality of first values based on the observation recorded from the corresponding digital interaction.

13. The system of claim 9, wherein the act (E) comprises:

determining if the count of values from the plurality of first values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least a selected threshold amount, wherein the attribute is determined to be anomalous in response to determining that the count of values from the plurality of first values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least the selected threshold amount.

14. The system of claim 9, wherein the method further comprises an act of:

(G) dividing the plurality of second values of the attribute into the plurality of buckets.

15. The system of claim 9, wherein the plurality of buckets comprises a plurality of first buckets, and wherein the method further comprises acts of:

(G) determining if the count of values from the plurality of first values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least a selected threshold amount; and

(H) in response to determining that the count of values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least the selected threshold amount, dividing the plurality of values into a plurality of second buckets, wherein there are more second buckets than first buckets.

16. The system of claim 9, wherein the method further comprises an act of:

(G) selecting a plurality of attributes, the plurality of attributes comprising the attribute, wherein acts (A)-(F) are performed for each attribute of the plurality of attributes.

17. A computer-implemented method for analyzing a plurality of digital interactions over a time period, the method comprising acts of:

(A) identifying a plurality of values of an attribute, each value of the plurality of values corresponding respectively to a digital interaction of the plurality of digital interactions;

(B) dividing the plurality of values into a plurality of first buckets;

(C) for at least one bucket of the plurality of first buckets, determining a count of values from the plurality of values that fall within the at least one bucket;

(D) comparing, by at least one processor, the count of values from the plurality of values that fall within the at least one bucket against historical information regarding the attribute, wherein;

the historical information regarding the attribute comprises an expected count for the at least one bucket; and

the count of values from the plurality of first values that fall within the at least one bucket is compared against the expected count for the at least one bucket;

(E) determining whether the attribute is anomalous over the time period, based at least in part on a result of the act (D), comprising;

determining if the count of values from the plurality of values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least a selected threshold amount; and

in response to determining that the count of values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least the selected threshold amount, dividing the plurality of values into a plurality of second buckets, wherein there are more second buckets than first buckets; and

(F) in response to determining that the attribute is anomalous over the time period, storing, in a profile, information regarding the attribute.

18. The method of claim 17, wherein:

each value of the plurality of values comprises a time measurement between a first point and a second point in the corresponding digital interaction; and

each bucket of the plurality of first buckets comprises a range of time measurements.

19. The method of claim 17, wherein:

the act (B) of dividing the plurality of values into a plurality of first buckets comprises applying a hash-modding operation to each value of the plurality of values; and

each bucket of the plurality of first buckets corresponds to a residue of the hash-modding operation.

20. The method of claim 17, further comprising acts of:

(G) recording a plurality of observations with respect to the attribute, each observation of the plurality of observations being recorded from a corresponding digital interaction of the plurality of digital interactions; and

(H) deriving each value of the plurality of values based on the observation recorded from the corresponding digital interaction.

21. The method of claim 17, wherein:

the plurality of digital interactions comprises a plurality of first digital interactions;

the time period comprises a first time period;

the plurality of values comprises a plurality of first values of the attribute;

the method further comprises an act of;

(G) dividing a plurality of second values of the attribute into the plurality of first buckets, each value of the plurality of second values corresponding respectively to a digital interaction of a plurality of second digital interactions;

the expected count for the at least one bucket comprises a count of values from the plurality of second values that fall within the at least one bucket;

the plurality of second digital interactions were observed from a second time period, the second time period having a same length as the first time period; and

the first time period occurs after the second time period.

22. The method of claim 17, further comprising an act of:

(G) selecting a plurality of attributes, the plurality of attributes comprising the attribute, wherein acts (A)-(F) are performed for each attribute of the plurality of attributes.

23. A system for analyzing a plurality of digital interactions over a time period, the system comprising:

at least one processor; and

at least one computer-readable storage device having stored thereon executable instructions which, when executed, program the at least one processor to perform a method comprising acts of;

(A) identifying a plurality of values of an attribute, each value of the plurality of values corresponding respectively to a digital interaction of the plurality of digital interactions;

(B) dividing the plurality of values into a plurality of first buckets;

(C) for at least one bucket of the plurality of first buckets, determining a count of values from the plurality of values that fall within the at least one bucket;

(D) comparing the count of values from the plurality of values that fall within the at least one bucket against historical information regarding the attribute, wherein;

the historical information regarding the attribute comprises an expected count for the at least one bucket; and

the count of values from the plurality of first values that fall within the at least one bucket is compared against the expected count for the at least one bucket;

(E) determining whether the attribute is anomalous over the time period, based at least in part on a result of the act (D), comprising;

determining if the count of values from the plurality of values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least a selected threshold amount; and

in response to determining that the count of values that fall within the at least one bucket exceeds the expected count for the at least one bucket by at least the selected threshold amount, dividing the plurality of values into a plurality of second buckets, wherein there are more second buckets than first buckets; and

(F) in response to determining that the attribute is anomalous over the time period, storing, in a profile, information regarding the attribute.

24. The system of claim 23, wherein:

each value of the plurality of values comprises a time measurement between a first point and a second point in the corresponding digital interaction; and

each bucket of the plurality of first buckets comprises a range of time measurements.

25. The system of claim 23, wherein:

the act (B) of dividing the plurality of values into a plurality of first buckets comprises applying a hash-modding operation to each value of the plurality of values; and

each bucket of the plurality of first buckets corresponds to a residue of the hash-modding operation.

26. The system of claim 23, wherein the method further comprises acts of:

(G) recording a plurality of observations with respect to the attribute, each observation of the plurality of observations being recorded from a corresponding digital interaction of the plurality of digital interactions; and

(H) deriving each value of the plurality of values based on the observation recorded from the corresponding digital interaction.

27. The system of claim 23, wherein:

the plurality of digital interactions comprises a plurality of first digital interactions;

the time period comprises a first time period;

the plurality of values comprises a plurality of first values of the attribute;

the method further comprises an act of;

(G) dividing a plurality of second values of the attribute into the plurality of first buckets, each value of the plurality of second values corresponding respectively to a digital interaction of a plurality of second digital interactions;

the expected count for the at least one bucket comprises a count of values from the plurality of second values that fall within the at least one bucket;

the plurality of second digital interactions were observed from a second time period, the second time period having a same length as the first time period; and

the first time period occurs after the second time period.

28. The system of claim 23, wherein the method further comprises an act of:

(G) selecting a plurality of attributes, the plurality of attributes comprising the attribute, wherein acts (A)-(F) are performed for each attribute of the plurality of attributes.