Real-time network monitoring and security

• US 20050108573A1
• Filed: 09/10/2004
• Published: 05/19/2005
• Est. Priority Date: 09/11/2003
• Status: Active Grant

• Current Assignee: BAE Systems Plc
• Original Assignee: Detica Limited (BAE Systems Plc)
• Inventors: Bennett, Mark Arwyn, Morris, Philip, Piggott, Alexander Colin, Garfield, David John Michael
• Granted Patent: US 8,364,833 B2
• Time in Patent Office: Days
• US Class Current: 726/4
• CPC Class Codes: G06F 11/3048 where the topology of the computing system or computing system component explicitly influences the monitoring activity, e.g. serial, hierarchical systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers

Abstract

There is provided a hardware device for monitoring and intercepting data packetized data traffic at full line rate. In preferred high bandwidth embodiments, full line rate corresponds to rates that exceed 100 Mbytes/s and in some cases 1000 Mbytes/s. Monitoring and intercepting software, alone, is not able to operate on such volumes of data in real-time. A preferred embodiment comprises: a data delay buffer (208) with multiple delay outputs (216); a search engine logic (210) for implementing a set of basic search tools that operate in real-time on the data traffic; a programmable gate array (206); an interface (212) for passing data quickly to software sub-systems; and control means for implementing software control of the operation of the search tools. The programmable gate array (206) inserts the data packets into the delay buffer (208), extracts them for searching at the delay outputs and formats and schedules the operation of the search engine logic (210). One preferred embodiment uses an IP co-processor as the search engine logic.

Claims

1. An apparatus for detecting a predetermined bit pattern in data traffic comprising a plurality of data packets, the data packets having a header portion and a payload portion, the apparatus comprising:

a processing logic for receiving the data traffic in real-time and for dividing the data packets in the data traffic into bit sequences; and

an Internet Protocol (IP) co-processor unit having a bit pattern storage memory array, in which one or more predetermined bit patterns are stored, the IP co-processor unit comprising means for identifying, from within at least the payload portion, each data packet that contains a bit sequence that matches a predetermined bit pattern entry within the memory array.

2. An apparatus as claimed in claim 1, wherein the bit pattern storage memory array stores bit patterns as ternary data.

3. An apparatus as claimed in claim 1, wherein the bit sequences supplied to the IP co-processor unit are consecutive sequences of bits from a single data packet.

4. An apparatus as claimed in claim 3, wherein at least one of the predetermined bit patterns comprises two subpatterns, the at least one bit pattern having a length L bits, each subpattern being stored as a separate bit pattern entry in the memory array;

wherein the identifying means compares the subpatterns with bit sequences from an end portion of each data packet, the portion being within L bits of the beginning or of the end of the data packet; and

wherein the identifying means identifies each data packet that contains a bit sequence that matches one or other of the two subpatterns.

5. An apparatus as claimed in claim 1, further comprising a control interface, the control interface being in communication with the processing logic, the interface conveying software instructions to the processing logic to alter the bit pattern entries in the IP co-processor unit.

6. An apparatus as claimed in claim 5, wherein the control interface is integral with the processing logic.

7. An apparatus as claimed in claim 1, wherein the bit pattern storage memory array of the IP co-processor unit includes a plurality of bit offset entries corresponding to one or more predetermined bit patterns.

8. An apparatus as claimed in claim 1, further comprising at least one further IP co-processor unit, the or each further IP co-processor unit operating upon bit sequences having a predetermined offset relative to the bit sequences searched by the IP co-processor unit.

9. An apparatus as claimed in claim 8, wherein the or each further IP co-processor unit is provided with an identical copy of the contents of the bit pattern storage memory array of the IP co-processor unit.

10. An apparatus as claimed in claim 1, the apparatus further comprising a delay buffer for storing the data traffic at full line rate,

wherein data streams that contain a data packet that is identified as having a bit sequence that matches the predetermined bit pattern are extracted from an output of the delay buffer and passed to a software application for further processing.

11. An apparatus for detecting a predetermined bit pattern in data traffic comprising a plurality of data streams of data packets, the apparatus comprising:

a processing logic for receiving the data traffic in real-time and for dividing the data traffic into bit sequences;

a search engine logic having a bit pattern storage memory array, in which one or more predetermined bit patterns are stored, the search engine logic comprising means for identifying each data packet that contains a bit sequence that matches a predetermined bit pattern entry within the memory array; and

a delay buffer for storing the data traffic at full line rate, the delay buffer having one or more outputs,

wherein data streams that contain a data packet that is identified as having a bit sequence that matches a predetermined bit pattern are extracted from an output of the delay buffer and passed to a software application for further processing.

12. An apparatus as claimed in claim 11, wherein the bit sequences supplied to the search engine logic are consecutive sequences of bits from a single data packet.

13. An apparatus as claimed in claim 12, wherein at least one of the predetermined bit patterns comprises two subpatterns, each subpattern being stored as a separate bit pattern entry in the memory array;

and

wherein the identifying means compares the subpatterns with the formatted bit sequences and identifies each data packet that contains a bit sequence that matches one or other of the two subpatterns.

14. An apparatus as claimed in claim 11, wherein the search engine logic comprises means for performing a protocol based search, in which target data packets are identified by virtue of the bit sequence of at least one data field within each data packet.

15. An apparatus as claimed in claim 14, wherein the data field is a header field, the header field including data relating to one of the group including:

address information;

to/from port number information; and

,

data packet type identifier information.

16. An apparatus as claimed in claim 11, further comprising a control interface, the control interface being in communication with the processing logic, the interface conveying software instructions to the processing logic to alter the bit pattern entries in the search engine logic.

17. An apparatus as claimed in claim 16, wherein the control interface is integral with the processing logic.

18. An apparatus as claimed in claim 11, wherein the bit pattern storage memory array of the search engine logic includes a plurality of bit offset entries corresponding to one or more predetermined bit patterns.

19. An apparatus as claimed in claim 11, further comprising at least one further search engine logic, the or each further search engine logic operating upon data sequences having a predetermined offset relative to the data sequences searched by the search engine logic.

20. An apparatus as claimed in claim 19, wherein the or each further search engine logic is provided with an identical copy of the contents of the bit pattern storage memory array of the search engine logic.

21. An apparatus as claimed in claim 11, wherein the search engine logic is an IP co-processor unit.

22. A system for detecting a predetermined bit pattern in input data traffic comprising a plurality of data packets, the system comprising a hardware component and a software component, the hardware component including:

a processing logic for receiving the data traffic in real-time and for dividing the data traffic into bit sequences; and

a search engine logic having a bit pattern storage memory array, in which one or more predetermined bit patterns are stored, the search engine logic comprising means for identifying each data packet that contains a bit sequence that matches the predetermined bit pattern entry within the memory array,

wherein data packets processed by the search engine logic of the hardware component and identified as containing a matching bit sequence are passed to an executable software application of the software component for further processing, the bandwidth of data traffic passed on to the executable software application of the software component being substantially less than the bandwidth of the data traffic input into the hardware component.

23. A system as claimed in claim 22, wherein the processing logic is programmable in a higher level programming language.

24. A system as claimed in claim 22, wherein the bit sequences supplied to the search engine logic of the hardware component are consecutive sequences of bits from a single data packet.

25. A system as claimed in claim 24, wherein at least one of the predetermined bit patterns comprises two subpatterns, each subpattern being stored as a separate bit pattern entry in the memory array;

and

wherein the identifying means compares the subpatterns with the bit sequences and identifies each data packet that contains a bit sequence that matches one or other of the two subpatterns.

26. A system as claimed in claim 22, wherein the search engine logic of the hardware component comprises means for performing a protocol based search, in which target data packets are identified by virtue of the bit sequence of at least one data field within each data packet.

27. A system as claimed in claim 26, wherein the data field is a header field, the header field including data relating to one of the group including:

address information;

to/from port number information; and

,

data packet type identifier information.

28. A system as claimed in claim 22, wherein the hardware component further comprises a control interface, the control interface being in communication with the processing logic, the interface conveying software instructions from the software component to the processing logic to alter the bit pattern entries in the search engine logic.

29. A system as claimed in claim 28, wherein the control interface is integral with the processing logic.

30. A system as claimed in claim 22, wherein the bit pattern storage memory array of the search engine logic includes a plurality of bit offset entries corresponding to one or more predetermined bit patterns.

31. A system as claimed in claim 22, wherein the hardware component further comprises at least one further search engine logic, the or each further search engine logic operating upon bit sequences having a predetermined offset relative to the bit sequences searched by the search engine logic.

32. A system as claimed in claim 31, wherein the or each further search engine logic is provided with an identical copy of the contents of the bit pattern storage memory array of the search engine logic.

33. A system as claimed in claim 22, wherein the hardware component of the system further comprises a delay buffer for storing the data traffic at full line rate, wherein data streams that contain a data packet that is identified as having a bit sequence that matches the predetermined bit pattern are extracted from an output of the delay buffer and passed to a software application for further processing.

34. A system as claimed in claim 22, wherein the or each search engine logic of the hardware component is an IP co-processor unit.

35. An apparatus for detecting predetermined bit patterns in data traffic comprising a plurality of data packets, the apparatus comprising:

a processing logic for receiving the data traffic in real-time and for dividing the data traffic into bit sequences;

a search engine logic having a bit pattern storage memory array, in which a plurality of predetermined bit patterns are stored, the search engine logic comprising means for identifying each data packet that contains a bit sequence that matches a predetermined bit pattern entry within the memory array; and

a delay buffer for storing the data traffic at full line rate, the delay buffer having an input and a plurality of outputs, each output corresponding to a delay of a respective duration,

wherein the search engine logic is adapted to execute different bit pattern search strategies across the data traffic, where data packets which match a particular search strategy are output at a respective output of the delay buffer for further processing.

36. An apparatus as claimed in claim 35, wherein the search engine logic is adapted to execute a series of bit pattern search strategies, with each successive strategy being dependent upon at least one preceding search strategy.

37. An apparatus as claimed in claim 36, wherein successive strategies are determined in dependence upon the results of at least one preceding search strategy.