Methods and apparatus to identify malicious activity in a network

• US 9,769,190 B2
• Filed: 11/17/2016
• Issued: 09/19/2017
• Est. Priority Date: 11/14/2013
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Coskun, Baris
• Primary Examiner(s): POWERS, WILLIAM S
• Application Number:
  US15/354,214
• Publication Number:
  US 20170070528A1
• Time in Patent Office: 306 Days
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 63/0254 Stateful filtering
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

Example network monitoring methods disclosed herein include iteratively adjusting respective weights assigned to respective types of network activity features for devices monitored in a network, the iterative adjusting to determine an output set of weights corresponding to ones of the types of network activity features indicative of malicious network activity. For example, the iterative adjusting is to (1) reduce a first distance calculated between a first pair of reference devices previously classified as being associated with malicious network activity, and (2) increase a second distance calculated between a first one of the pair of the reference devices and a first unclassified device. Disclosed example network monitoring methods also include determining whether a second unclassified device is associated with malicious network activity based on the output set of weights.

Claims

1. A network monitor comprising:

memory including computer readable instructions; and

a processor to execute the computer readable instructions to perform operations including;

iteratively adjusting respective weights assigned to respective types of network activity features for devices monitored in a network, the iterative adjusting to determine an output set of weights corresponding to ones of the types of network activity features indicative of malicious network activity, the iterative adjusting to (1) reduce a first distance calculated between a first pair of reference devices selected from a first set of the devices previously classified as being associated with malicious network activity, and (2) increase a second distance calculated between a first one of the pair of the reference devices and a first unclassified device selected from a second set of the devices that are unclassified; and

determining whether a second unclassified device selected from the second set of the devices is associated with malicious network activity based on the output set of weights.

2. The network monitor of claim 1, wherein the types of network activity features include a first tier of network activity features generated from network log records obtained for the devices monitored in the network, a second tier of network activity features generated from the first tier of network activity features and the network log records, and a third tier of features generated from the second tier of network activity features.

3. The network monitor of claim 2, wherein the operations further include:

processing the network log records to determine a first set of network activity features for a first one of the devices monitored in the network, the first set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features; and

processing the network log records to determine a second set of network activity features for a second one of the devices monitored in the network, the second set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features.

4. The network monitor of claim 1, wherein the iterative adjusting further includes:

evaluating a squared weighted Euclidean distance function based on the weights, a first set of network activity features determined for the first one of the pair of the reference devices and a second set of network activity features determined for the second one of the pair of the reference devices to calculate the first distance; and

evaluating the squared weighted Euclidean distance function based on the weights, the first set of network activity features determined for the first one of the pair of the reference device and a third set of network activity features determined for the first unclassified device to calculate the second distance.

5. The network monitor of claim 4, wherein the iterative adjusting of the respective weights includes adjusting the respective weights according to a stochastic gradient descent algorithm.

6. The network monitor of claim 1, wherein the determining of whether the second unclassified device is associated with malicious network activity includes:

determining a third distance based on the output set of weights and a respective set of network activity features determined for the second unclassified device; and

comparing the third distance to a threshold to determine whether the second unclassified device is associated with malicious network activity.

7. A network monitoring method comprising:

iteratively adjusting, by executing an instruction with a processor, respective weights assigned to respective types of network activity features for devices monitored in a network, the iterative adjusting to determine an output set of weights corresponding to ones of the types of network activity features indicative of malicious network activity, the iterative adjusting to (1) reduce a first distance calculated between a first pair of reference devices selected from a first set of the devices previously classified as being associated with malicious network activity, and (2) increase a second distance calculated between a first one of the pair of the reference devices and a first unclassified device selected from a second set of the devices that are unclassified; and

determining, by executing an instruction with the processor, whether a second unclassified device selected from the second set of the devices is associated with malicious network activity based on the output set of weights.

8. The method of claim 7, wherein the types of network activity features include a first tier of network activity features generated from network log records obtained for the devices monitored in the network, a second tier of network activity features generated from the first tier of network activity features and the network log records, and a third tier of features generated from the second tier of network activity features.

9. The method of claim 8, further including:

processing the network log records to determine a first set of network activity features for a first one of the devices monitored in the network, the first set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features; and

processing the network log records to determine a second set of network activity features for a second one of the devices monitored in the network, the second set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features.

10. The method of claim 7, wherein the iterative adjusting further includes:

evaluating a squared weighted Euclidean distance function based on the weights, a first set of network activity features determined for the first one of the pair of the reference devices and a second set of network activity features determined for the second one of the pair of the reference devices to calculate the first distance; and

evaluating the squared weighted Euclidean distance function based on the weights, the first set of network activity features determined for the first one of the pair of the reference device and a third set of network activity features determined for the first unclassified device to calculate the second distance.

11. The method of claim 10, wherein the iterative adjusting of the respective weights includes adjusting the respective weights according to a stochastic gradient descent algorithm.

12. The method of claim 7, wherein the determining of whether the second unclassified device is associated with malicious network activity includes:

determining a third distance based on the output set of weights and a respective set of network activity features determined for the second unclassified device; and

comparing the third distance to a threshold to determine whether the second unclassified device is associated with malicious network activity.

13. The method of claim 7, further including selecting the pair of the reference devices and the first unclassified device based on a random number generator.

14. A tangible computer readable storage medium including computer readable instructions which, when executed, cause a processor to perform operations comprising:

iteratively adjusting respective weights assigned to respective types of network activity features for devices monitored in a network, the iterative adjusting to determine an output set of weights corresponding to ones of the types of network activity features indicative of malicious network activity, the iterative adjusting to (1) reduce a first distance calculated between a first pair of reference devices selected from a first set of the devices previously classified as being associated with malicious network activity, and (2) increase a second distance calculated between a first one of the pair of the reference devices and a first unclassified device selected from a second set of the devices that are unclassified; and

determining whether a second unclassified device selected from the second set of the devices is associated with malicious network activity based on the output set of weights.

15. The tangible computer readable storage medium of claim 14, wherein the types of network activity features include a first tier of network activity features generated from network log records obtained for the devices monitored in the network, a second tier of network activity features generated from the first tier of network activity features and the network log records, and a third tier of features generated from the second tier of network activity features.

16. The tangible computer readable storage medium of claim 15, wherein the operations further include:

processing the network log records to determine a first set of network activity features for a first one of the devices monitored in the network, the first set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features; and

processing the network log records to determine a second set of network activity features for a second one of the devices monitored in the network, the second set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features.

17. The tangible computer readable storage medium of claim 14, wherein the iterative adjusting further includes:

evaluating a squared weighted Euclidean distance function based on the weights, a first set of network activity features determined for the first one of the pair of the reference devices and a second set of network activity features determined for the second one of the pair of the reference devices to calculate the first distance; and

evaluating the squared weighted Euclidean distance function based on the weights, the first set of network activity features determined for the first one of the pair of the reference device and a third set of network activity features determined for the first unclassified device to calculate the second distance.

18. The tangible computer readable storage medium of claim 17, wherein the iterative adjusting of the respective weights includes adjusting the respective weights according to a stochastic gradient descent algorithm.

19. The tangible computer readable storage medium of claim 14, wherein the determining of whether the second unclassified device is associated with malicious network activity includes:

determining a third distance based on the output set of weights and a respective set of network activity features determined for the second unclassified device; and

comparing the third distance to a threshold to determine whether the second unclassified device is associated with malicious network activity.

20. The tangible computer readable storage medium of claim 14, wherein the operations further include selecting the pair of the reference devices and the first unclassified device based on a random number generator.