Method and apparatus of detecting network activity

• US 20060265745A1
• Filed: 07/17/2002
• Published: 11/23/2006
• Est. Priority Date: 07/26/2001
• Status: Abandoned Application

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Shackleton, Mark Andrew, Hodgson, Paul W.
• Application Number:
  US10/483,068
• Publication Number:
  US 20060265745A1
• Time in Patent Office: Days
• US Class Current: 726/22
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms

Abstract

Embodiments of the invention are concerned with a method of, and apparatus for, identifying types of network behaviour for use in identifying aberrant network behaviour. In particular, embodiments are concerned with identifying email viruses. The method comprises the steps of: collecting data representative of network traffic that has travelled over a network; training a classification means to recognise a plurality of network behaviour types from the collected data; and for unseen data travelling over the network, classifying the unseen data into one of the defined network behaviour types.

Claims

1. A method of identifying behaviour patterns in respect of a system that operates over a communications network, the system comprising a plurality of server computers and client computers, wherein at least some of the server computers are arranged to deliver data to, and receive data from, one or more client computers over the communications network, the method comprising the steps of:

(a) receiving data in respect of data which have been sent within the system, each of the received data items identifying the computer, within the system, to and/or from which the said data item has been sent;

(b) organising the received data into a representation indicative of the distribution of data sent within the system, as a function of identified computer; and

(c) using the representation to train a classification means to recognise a plurality of behaviour types.

2. A method according to claim 1, including transforming the representation into a format suitable for input into the classification means.

3. A method according to claim 1, in which step (b) comprises creating a topological representation of the server and client computers in the system, and the method includes, for each received data item, incrementing a counter representative of a level of activity associated with the identified computer;

and

adding an identifier, which is indicative of a level of activity, to whichever part of the topological representation corresponds to the identified computer, thereby creating a representation indicative of a distribution of data sent within the system.

4. A method according to claim 3, in which said topological representation comprises

a plurality of regions, each of which is representative of an area of the network,

a plurality of sub-regions, each of which is representative of a server computer within a corresponding area of the network, and

a plurality of sub-sub regions, each of which is representative of a client computer acting as a client to a corresponding server computer;

and

in which the step of adding an identifier to whichever part of the topological representation corresponds to the identified computer involves adding an identifier to whichever sub region or sub-sub region corresponds thereto.

5. A method according to claim 3, in which the level of activity is normalised over the topological representation.

6. A method according to claim 5, wherein the transforming step comprises transforming the representation into a frequency representation of activity;

and converting the frequency representation into a vector, which vector is suitable for input into a classification means.

7. A method according to claim 6, in which the step of transforming the representation into a frequency representation comprises applying a Fourier transform to the said representation.

8. A method according to claim 6, in which the step of converting the frequency representation into a vector comprises sampling the frequency representation in order to extract vector values corresponding thereto.

9. A method according to claim 1, in which the received data items additionally identify attributes of the data sent within the system, and in which step (b) comprises the steps of

creating a plurality of lists, each of which corresponds to a link between server machines in the system;

and

for each received data item;

identifying a link over which the corresponding sent data item has passed;

identifying a list corresponding to the identified link;

identifying attributes of the data item; and

for each identified attribute, incrementing a counter corresponding thereto in the identified list.

10. A method according to claim 9 when dependent on claim 2, in which the received data items additionally identify attributes of the data sent within the system, and in which step (b) comprises the steps of

creating a Plurality of lists, each of which corresponds to a link between server machines in the system;

and

for each received data item;

identifying a link over which the corresponding sent data item has passed;

identifying a list corresponding to the identified link;

identifying attributes of the data item; and

for each identified attribute, incrementing a counter corresponding thereto in the identified list; and

the transforming step comprises creating a vector comprising at least some of the lists, which vector is suitable for input into a classification means.

11. A method according to claim 2, in which data is received in respect of a plurality of time periods, and the organising and transforming steps are performed for the said plurality of the said time periods, thereby generating a plurality of transformed representations for inputting to the classification means.

12. A method according to claim 11, in which the method is carried out for a plurality of different size time periods, so that there are a plurality of behaviour types for each size of time period.

13. A method according to claim 14, in which, for each size time period, a different respective classification means is used.

14. A method according to claim 1, wherein the data being received is email data, and the aberrant behaviour to be identified is email viruses propagating through the network.

15. A method according to claim 14, in which the receiving step (a) includes collecting data from any one of a log file being part of a firewall arrangement, or a log file accessible from an email server machine, or a plurality of log files accessible from a plurality of email server machines.

16. A method according to claim 1, including arranging the received data into groups of received data as a function of type of sent data.

17. A method of identifying aberrant behaviour in respect of unseen data items that have been sent within a system comprising a plurality of server computers and client computers, including the steps of

receiving data in respect of the unseen data items;

organising the received data into a representation according to claim 3;

transforming the representation into a format suitable for input into the classification means in which the level of activity is normalized over the topological representation and wherein the transforming step comprises transforming the representation into a frequency representation of activity; and

converting the frequency representation into a vector, which vector is suitable for input into a classification means;

inputting the transformed representation to the trained classification means; and

operating the classification means in order to classify the unseen data as a type of a behaviour.

18. Apparatus for identifying aberrant behaviour in respect of a system that operates within a communications network, the system comprising a plurality of server computers and client computers, wherein each server computer is arranged to deliver data to, and receive data from, one or more client computers over the communications network, the apparatus comprising

receiving means arranged to receive data in respect of data which have been sent within the system, each of the received data items identifying the computer, within the system, from and/or to which the said data item has been sent during a time period;

means operable to arrange the received data into groups of received data as a function of type of sent data, so that each group represents a type of behaviour;

organising means arranged to organise data in each group into a representation indicative of a distribution of data sent within the system, as a function of identified computer during the period; and

a classification means operable to receive the representation as input and operable to generate an output representative of a behaviour corresponding to the group.

19. Apparatus according to claim 18, including transforming means arranged to transform the representation into a format suitable for input into a classification means.

20. Apparatus according to claim 19, wherein the receiving means is in operative association with means operable to retrieve data from any one of a log file being part of a firewall arrangement, or a log file accessible from a server machine, or a plurality of log files accessible from a plurality of server machines.

21. Apparatus according to claim 19 or claim 20, wherein the organising means comprises means arranged to create a representation indicative of a level of activity of server and client computers of the system.

22. Apparatus according to claim 21, wherein the transforming means includes means operable transform the representation into a frequency representation.

23. Apparatus according to claim 22, wherein the transforming means includes means operable to apply a Fourier transform to the representation, thereby generating the frequency representation.

24. Apparatus according to claim 18, including means arranged to analyse data passing through at least some of the server computers and to identify attributes associated with the analysed data, wherein received data in respect of the analysed data identifies the said server computer and identified attributes.

25. Apparatus according to claim 18, wherein the classification means comprises any one of a neural network, a statistical classifier or a pattern recogniser.

26. Apparatus according to claim 25, wherein, when the classification means comprises a neural network, the said neural network comprising at least

an input layer comprising a plurality of input nodes,

a hidden layer comprising a plurality of hidden nodes, which hidden layer is in operative association with the input layer, and

an output layer comprising a plurality of output nodes, which output layer is in operative association with the hidden layer,

wherein each of the output nodes corresponds to a type of behaviour.

27. Apparatus according to claim 18, wherein the received data is email data, and the aberrant behaviour to be identified is email viruses propagating through the network.

28. Apparatus according to claim 27, wherein at least some of the output nodes correspond to rates of email virus propagation.

29. Apparatus according to claim 18, further including alerting means arranged in operative association with at least some of the output nodes and operable to generate one of a plurality of alert outputs in dependence on activation of output nodes.

30. An email activity device for use in identifying email viruses, the device being located in a network and operable to communicate with other devices in the network, comprising

retrieving means operable to retrieve data representative of email traffic, during a time period, from any one of:

a log file being part of a firewall arrangement, or a log file accessible from an email server machine, or a plurality of log files accessible from a plurality of email server machines;

organising means arranged to organise the retrieved data into a representation indicative of a distribution of the said email traffic during the period;

transforming means arranged to transform the representation into a format suitable for input into a classification means; and

a classification means operable to receive the transformed representation as input and operable to generate an output representative of a type of email traffic.