Network monitor internals description

• US 20040030796A1
• Filed: 06/10/2003
• Published: 02/12/2004
• Est. Priority Date: 06/16/2000
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: Securify, Inc. (McAfee, LLC)
• Inventors: Cooper, Geoffrey, Sherlock, Kieran Gerard, Shaw, Robert Allen, Valente, Luis Filipe Pereira
• Granted Patent: US 7,272,646 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: H04L 41/0604 using filtering, e.g. reduction of information by using priority, element types, position or time
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/5012 determining service availability, e.g. which services are available at a certain point in time
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/067 using time frame reporting
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 43/18 Protocol analysers
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/0263 Rule management
• CPC Class Codes: H04L 63/0823 using certificates cryptographic mechanisms or cryptographic arrangements for entity authentication involving certificates H04L9/3263
• CPC Class Codes: H04L 63/083 using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 Vulnerability analysis
• CPC Class Codes: H04L 63/166 at the transport layer
• CPC Class Codes: H04L 69/22 : Parsing or analysis of headers

Abstract

A method and apparatus for a network monitor internals mechanism that serves to translate packet data into multiple concurrent streams of network event data is provided. The data translation is accomplished by interpreting both sides of each protocol transaction.

Claims

1. An apparatus for translating packet data into a plurality of serialized streams of network event data, comprising:

means for accepting said packet data live or from a file containing said packet data;

means for extracting security-sensitive information from said packet data; and

means for generating output in said serialized streams of network event data, wherein said network event data is in encoded format.

2. The apparatus of claim 1, wherein said encoded format is DME encoded format.

3. The apparatus of claim 1, wherein said generated output is suitable for processing by, but not limited to means for logging, means for debugging, and a policy engine.

4. The apparatus of claim 1, said means for accepting said packet data, further comprising:

a packet structure for representing said packet data;

said packet structure comprising a stack-based structure, said stack-based structure comprising a plurality of protocol headers and protocol trailers, a generic source address location, a generic destination address location, and a plurality of flags; and

a connection structure representing a connection, said connection structure comprising protocol data to be arranged in a stack allocation, said stack representing access across corresponding protocol layer boundaries, said connection structure further comprising a plurality of modules corresponding to said protocol data, wherein each of said modules implements a generic series of operations;

wherein said headers and trailers are marked for allowing ease of focus on a protocol layer of said corresponding protocol layer boundaries, and wherein said address locations and said plurality of flags are used to pass information up and down said stack.

5. The apparatus of claim 4, wherein said boundaries comprise, but are not limited to network, transport, session, application, and transaction boundaries.

6. The apparatus of claim 4, wherein said generic operations comprise, but are not limited to the following operations:

initializing;

binding a connection to the parameters of a connection as represented by a packet;

processing a packet as input into to a connection;

giving back a packet to a lower layer of protocol for consumption at a later time when additional data has arrived;

getting more in a connection of an amount needed of data from a client or server from a higher layer of protocol to a lower layer;

stopping collecting data for a connection;

a client, server, or both shutting down a connection;

deleting an instance of a protocol from a connection;

signaling an alarm has expired on a connection at an official time; and

reversing the choice of client and server for a connection.

7. The apparatus of claim 4, further comprising:

means for using a “

stopped collecting”

state where said packets may be ignored except to use said headers to determine a connection state associated with said connection and freeing resources when said connection terminates, thereby increasing efficiency in said translating packet data.

8. The apparatus of claim 7, wherein said means for using headers and freeing resources is implemented as a hardware filter to stop additional packets of data from arriving.

10. The apparatus of claim 1, wherein when said accepting packet data is live, current time is time said packet data is received.

11. The apparatus of claim 1, wherein when said accepting packet data is from said file, further comprising means for processing said packet data relative to time and time intervals of when said packet was originally received into the file.

12. The apparatus of claim 1, further comprising:

means for setting an alarm dependent on a predetermined value of current time.

13. The apparatus of claim 12, wherein means for setting an alarm further comprises means for setting an alarm on every protocol layer of a connection, from lowest layer to highest layer.

14. The apparatus of claim 1, further comprising:

a connectionless association feature, whereby IP host communication is detected.

15. The apparatus of claim 14, wherein said association feature treats associations as connections.

16. The apparatus of claim 1, said means for generating output further comprising:

means for establishing a transaction, said transaction comprising an association between a network monitor connection and a network event.

17. The apparatus of claim 16, said means for generating output further comprising:

an interface for communicating with a policy engine, said interface comprising calls for connecting and for starting and finishing a plurality of transactions, and event specific calls.

18. The apparatus of claim 17, wherein said plurality of transactions are active at the same time.

19. The apparatus of claim 17, said means for generating output further comprising:

means for translating said interface calls into generic output calls used to filter output data.

20. The apparatus of claim 19, wherein generic output calls comprise, but are not limited to:

NULL;

acts as an endpoint, but discards input data without doing anything;

SM;

connects by procedure call directly to policy processing;

ENC;

generate encoded form of output; and

LOG;

generate textual form of output.

21. The apparatus of claim 1, further comprising:

means for decoding encoded versions of network events.

22. The apparatus of claim 1, wherein said encoded format comprises:

a header;

embedded agent descriptors indicating location of source of said packet data;

type map used in detecting update types not supported by old software, thereby; and

encoded transactions.

23. The apparatus of claim 1, wherein different protocol layers of said network event data are combined to facilitate understanding of events in said packet data.

9. The apparatus of claim 9, further comprising:

means for using time received of said packet data as a point of reference and using corresponding time intervals for aligning results, said results stored in a database.

24. A method for translating packet data into a plurality of serialized streams of network event data, comprising:

accepting said packet data live or from a file containing said packet data;

extracting security-sensitive information from said packet data; and

generating output in said serialized streams of network event data, wherein said network event data is in encoded format.

25. The method of claim 24, wherein said encoded format is DME encoded format.

26. The method of claim 24, wherein said generated output is suitable for processing by, but not limited to logging, debugging, and a policy engine.

27. The method of claim 24, said accepting said packet data, further comprising:

providing a packet structure for representing said packet data, said packet structure comprising a stack-based structure, said stack-based structure comprising a plurality of protocol headers and protocol trailers, a generic source address location, a generic destination address location, and a plurality of flags; and

providing a connection structure representing a connection, said connection structure comprising protocol data to be arranged in a stack allocation, said stack representing access across corresponding protocol layer boundaries, said connection structure further comprising a plurality of modules corresponding to said protocol data, wherein each of said modules implements a generic series of operations;

wherein said headers and trailers are marked for allowing ease of focus on a protocol layer of said corresponding protocol layer boundaries, and wherein said address locations and said plurality of flags are used to pass information up and down said stack.

28. The method of claim 27, wherein said boundaries comprise, but are not limited to network, transport, session;

application, and transaction boundaries.

29. The method of claim 27, wherein said generic operations comprise, but are not limited to the following operations:

initializing;

binding a packet and a connection;

processing a packet as input into to a connection;

giving back a packet to a higher layer of protocol for later transmission;

getting more in a connection of an amount needed of data from a client or server from a higher layer of protocol to a lower layer;

stopping collecting data for a connection;

a client, server, or both shutting down a connection;

deleting an instance of a protocol from a connection;

signaling an alarm has expired on a connection at an official time; and

reversing the choice of client and server for a connection.

30. The method of claim 27, further comprising:

providing a “

stopped collecting”

state whereby said packets may be ignored except to use said headers to determine a connection state associated with said connection and free resources when said connection terminates, thereby increasing efficiency in said translating packet data.

31. The method of claim 30, wherein using headers and freeing resources is implemented as a hardware filter to stop additional packets of data from arriving.

32. The method of claim 24, further comprising:

using time received of said packet data as a point of reference and using corresponding time intervals for aligning results, said results stored in a database.

33. The method of claim 24, wherein when said accepting packet data is live, current time is time said packet data is received.

34. The method of claim 24, wherein when said accepting packet data is from said file, further comprising processing said packet data relative to time and time intervals of when said packet was originally received into the file.

35. The method of claim 24, further comprising:

setting an alarm dependent on a predetermined value of current time.

36. The method of claim 35, wherein setting an alarm further comprises setting an alarm on every protocol layer of a connection, from lowest layer to highest layer.

37. The method of claim 24, further comprising:

providing a connectionless association feature, whereby IP host communication is detected.

38. The method of claim 37, wherein said association feature treats associations as connections.

39. The method of claim 24, said generating output further comprising:

establishing a transaction, said transaction comprising an association between a network monitor connection and a network event.

40. The method of claim 39, said generating output further comprising:

providing an interface for communicating with a policy engine, said interface comprising calls for connecting and for starting and finishing a plurality of transactions, and event specific calls.

41. The method of claim 40, wherein said plurality of transactions are active at the same time.

42. The method of claim 40, said generating output further comprising:

translating said interface calls into generic output calls used to filter output data.

43. The method of claim 42, wherein generic output calls comprise, but are not limited to:

NULL;

acts as an endpoint, but discards input data without doing anything;

SM;

connects by procedure call directly to policy processing;

ENC;

generate encoded form of output; and

LOG;

generate textual form of output.

44. The method of claim 24, further comprising:

decoding encoded versions of network events.

45. The method of claim 24, wherein said encoded format comprises:

a header;

embedded agent descriptors indicating location of source of said packet data;

type map used in detecting update types not supported by old software, thereby; and

encoded transactions.

46. The method of claim 24, wherein different protocol layers of said network event data are combined to facilitate understanding of events in said packet data.