Flow logging for connection-based anomaly detection

• US 20050286423A1
• Filed: 06/28/2004
• Published: 12/29/2005
• Est. Priority Date: 06/28/2004
• Status: Active Grant

• Current Assignee: Riverbed Technology, LLC (Riverbed Technology Incorporated)
• Original Assignee: Riverbed Technology Incorporated
• Inventors: Kohler, Edward W. Jr., Ratin, Andrew, Poletto, Massimiliano Antonio
• Granted Patent: US 7,929,534 B2
• Time in Patent Office: Days
• US Class Current: 370/235
• CPC Class Codes: H04L 41/0233 Object-oriented techniques, for representation of network management data, e.g. common object request broker architecture [CORBA]
• CPC Class Codes: H04L 41/06 Management of faults, events, alarms or notifications
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 43/12 Network monitoring probes
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

A plurality of flow collector devices is disposed to collect flow information on a network. Duplicate flow records received from the flow collectors are eliminated by determining whether a pair of flow records has the same, source and destination flow identifiers and were received within a predefined time-period. Non-duplicated flow records received from the plurality of flow collector devices are stored and used to produces a connection table that maps each node on the network to a record that stores information about traffic to or from the node from non-duplicated flow records. The connection table stores statistical information of packets on the network based on a time-slice basis.

Claims

1. A method, comprising:

collecting flow records from a plurality of flow collector devices that are disposed to collect flow information on a network;

determining whether a pair of flow records has the same source and destination flow identifiers and was received within a predefined time period to eliminate duplicate flow records received from the flow collectors; and

storing remaining, non duplicated flow records received from the plurality of flow collector devices.

2. The method of claim 1 further comprising

producing from non duplicated flow records a connection table that maps each node on the network to a record that stores information about traffic to or from the node.

3. The method of claim 1 further comprising

producing a connection table from non duplicated flow records, the connection table storing statistical information of packets on the network based on a time-slice basis.

4. The method of claim 1 wherein the time-period is about a time slice.

5. The method of claim 1 wherein determining whether a pair of flow records have the same source and destination flow identifiers includes determining whether two flows have the same source and destination addresses.

6. The method of claim 1 wherein determining whether a pair of flow records have the same source and destination flow identifiers includes determining whether two flows have the same source and destination ports.

7. The method of claim 1 determining whether a pair of flow records has the same source and destination flow identifiers includes determining whether two flows have the same protocol.

8. The method of claim 1 wherein determining whether a pair of flow records has the same source and destination flow identifiers includes determining whether two flows have the same protocol.

9. The method of claim 1 wherein the source and destination statistics include determining whether two flows have the same source and destination addresses, source and destination ports and protocol.

10. The method of claim 2 wherein the connection table includes a plurality of records that are indexed by time.

11. The method of claim 2 wherein the connection table includes a plurality of records that are indexed by source address, destination address and time.

12. A system comprises:

a computing device including a computer readable medium storing a computer program that includes instructions to cause the computing device to;

collect flow records from a plurality of flow collector devices that are disposed to collect flow information on a network;

determine whether a pair of flow records has the same whether a pair of flow records have the same source and destination flow identifiers and whether the records were received within a predefined time period to eliminate duplicate flow records received from the flow collectors;

store remaining, non duplicated flow records received data from the plurality of collector devices.

13. The system of claim 12 further comprising instructions to:

produce a connection table from non duplicated flow records, the connection table mapping each node on the network to a record that stores information about traffic to or from the node.

14. The system of claim 12 further comprising instructions to:

producing a connection table from the non duplicated flow records that stores statistical information of packets on the network based on a time-slice basis.

15. The system of claim 12 wherein the time-period is about a time slice.

16. The system of claim 12 wherein instructions to determine whether a pair of flow records has the same source and destination flow identifiers includes instructions to determine whether the two flows have the same source and destination hosts, source and destination ports and protocol.

17. A computer readable medium storing a computer program that includes instructions to cause a computing device to:

collect flow records from a plurality of flow collector devices that are disposed to collect flow information on a network;

determine whether a pair of flow records has the same source and destination flow identifiers and whether the records were received within a predefined time period to eliminate duplicate flow records received from the flow collectors;

store remaining, non duplicated flow records received data from the plurality of collector devices.

18. The computer program product of claim 17 further comprising instructions to:

produce a connection table from non duplicated flow records, the connection table mapping each node on the network to a record that stores information about traffic to or from the node.

19. The computer program product of claim 17 further comprising instructions to:

produce a connection table from the non duplicated flow records that stores statistical information of packets on the network based on a time-slice basis.

20. The computer program product of claim 17 wherein the time-period is about a time slice.

21. The computer program product of claim 17 wherein instructions to determine whether flows have the source and destination flow identifiers include instructions to determine whether the two flows have the same source and destination hosts, source and destination ports and protocol.