System and method for real-time analysis of network traffic

• US 9,210,061 B2
• Filed: 01/14/2015
• Issued: 12/08/2015
• Est. Priority Date: 09/13/2013
• Status: Active Grant

• Current Assignee: Network Kinetix, Llc
• Original Assignee: Network Kinetix, Llc
• Inventors: Richards, Carissa, Quoc, Myvan, Coddington, Neal, McCarthy, George, Ramachandran, Hariharan
• Primary Examiner(s): Katsikis, Kostas
• Application Number:
  US14/596,781
• Publication Number:
  US 20150156098A1
• Time in Patent Office: 328 Days
• Field of Search: 709/224, 709/223
• US Class Current: 1/1
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5051 Service on demand, e.g. definition and deployment of services in real time
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/0894 Packet rate
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04M 15/47 Fraud detection or prevention means
• CPC Class Codes: H04M 15/8214 Data or packet based
• CPC Class Codes: H04M 15/8228 Session based
• CPC Class Codes: H04M 3/2218 Call detail recording
• CPC Class Codes: H04W 12/128 Anti-malware arrangements, e.g. protection against SMS fraud or mobile malware

Abstract

A mirrored live-data flow of the live-data flow passing through a selected point within a network is monitored at a first processing node. The live-data flow comprises data that is in active transmission between endpoints in the network and prior to exit from the network and onward storage of the data in a database. Each packet within the mirrored data flow is decoded at the first processing node according to each protocol associated with a packet. Packets having a plurality of protocols associated therewith are decoded in parallel with each other. Each of the decoded packets are compared at the first processing node to a set of predetermined or deduced conditions. A predetermined or deduced response is executed based upon detection of a predetermined or deduced condition within the decoded packets. At least a portion of the decoded packets of the live-data flow causing execution of the predetermined or deduced response are processed at a second processing node to determine a manner for controlling an operation of the network at a same time the live-data flow is in active transmission between the endpoints in the network. The operation of the network is controlled in response to the processing step.

Claims

1. A method for monitoring live-data flow through a network, comprising:

monitoring, at a first processing node, a mirrored live-data flow of the live-data flow passing through a selected point within the network in a non-intrusive manner that does not affect the live-data flow passing through the selected point, wherein the live-data flow comprises data that is in active transmission between endpoints in the network and prior to onward storage of the data in a database;

decoding, at the first processing node, each packet within the mirrored data flow according to each protocol associated with a packet, wherein packets have a plurality of protocols associated therewith are decoded in parallel with each other;

comparing, at the first processing node, each of the decoded packets to at least one of a set of predetermined or deduced conditions received from a second processing node;

executing at least one of a predetermined or deduced response including an indication of occurrence of the at least one predetermined or deduced condition based upon detection of the at least one predetermined or deduced condition within the decoded packets;

processing, at the second processing node, at least a portion of the decoded packets of the live-data flow causing execution of the at least one predetermined or deduced response to determine a manner for controlling an operation of the network at a same time the live-data flow is in active transmission between the endpoints in the network; and

controlling the operation of the network in response to the processing step while events associated with the live-data flow are occurring within the network.

2. The method of claim 1, wherein the step of decoding further includes the steps of:

determining whether the decoded packets are wanted for further processing;

discarding unwanted decoded packets;

assigning wanted decoded packets to at least one time dependent buffer for comparison to the at least one predetermined or deduced conditions, the time dependent buffer comprising a variable length bit map;

forwarding wanted packets to the assigned at least one time dependent buffer for comparison to the set of at least one predetermined or deduced conditions.

3. The method of claim 2 further including the step of transmitting the wanted data packets to the second processing node for the step of processing.

4. The method of claim 1, wherein the step of processing further comprises the steps of:

receiving the indication of detection of one of the at least one predetermined or deduced conditions;

assigning at least one predefined application associated with the indication;

processing the decoded data flow using the assigned at least one predefined application in simultaneous multithreaded parallel fashion to generate at least one control action for controlling the operation in the network at the same time the live-data flow is in active transmission between the endpoints in the network, wherein controlling comprises alerting or instructing a network provider to affect an event outcome before the event is finalized; and

controlling the operation of the network using the at least one control action.

5. The method of claim 1, wherein the step of processing further comprises processing the portion of the decoded packets decoded data flow using relational language processing.

6. The method of claim 1, wherein the step of executing the at least one predetermined or deduced response further comprises generating an alert to the network provider in response to the at least one predetermined or deduced condition.

7. The method of claim 1, wherein the step of processing further comprises the step of processing, at the second node at least a portion of the decoded packets of the live-data flow, the processing causing execution of the predetermined or deduced response and the deducing of dynamically created conditions responsive to historical data, the dynamically deduced conditions established as part of a learning process including statistical models and pattern recognition, operating within the second node with the option to provide the dynamically deduced conditions to the first processing node to update the deduced conditions.

8. The method of claim 1, wherein the step of controlling further comprises maintaining at least one network operating parameter at a predetermined level and generating an alarm when the live-data of the data flow indicates that the at least one network parameter is moving from the predetermined level.

9. The method of claim 1, wherein the step of controlling further comprises detecting an occurrence of a fraudulent activity on the network based on the live-data within the data flow and generating a warning to notify a network provider of the fraudulent activity as the fraudulent activity is being transmitted in the data flow over the network.

10. A system for monitoring live-data flow through a network, comprising:

a server communicating with the network;

a network interface card associated with the server for providing access to the data flow through the network;

a processor within the server the processor implementing a first processing node for;

monitoring, at a first processing node, a mirrored live-data flow of the live-data flow passing through a selected point within the network in a non-intrusive manner that does not affect the live-data flow passing through the selected point, wherein the live-data flow comprises data that is in active transmission between endpoints in the network and prior to storage of the data in a database;

decoding, at the first processing node, each packet within the mirrored data flow according to each protocol associated with a packet, wherein packets have a plurality of protocols associated therewith are decoded in parallel with each other;

comparing, at the first processing node, each of the decoded packets to at least one of a set of predetermined or deduced conditions;

executing at least one of a predetermined or deduced response including an indication of occurrence of the at least one predetermined or deduced condition based upon detection of at least one predetermined or deduced condition within the decoded packets;

the processor within the server the processor further implementing a second processing node for;

processing, at a second processing node, at least a portion of the decoded packets of the live-data flow causing execution of the at least one predetermined or deduced response to determine a manner for controlling an operation of the network at a same time the live-data flow is in active transmission between the endpoints in the network, wherein controlling comprises alerting or instructing a network provider to affect an event outcome before the event is finalized; and

controlling the operation of the network in response to the processing step while events associated with the live-data flow are occurring within the network.

11. The system of claim 10, further including a memory associated with the second processing node, the memory including application processing rule sets each defining a different manner for controlling operations at the same time the live-data flow is in active transmission between the endpoints in the network in response to the at least one detected predetermined or deduced condition, wherein controlling comprises alerting or instructing a network provider to affect an event outcome before the event is finalized.

12. The system of claim 11, wherein the memory further defines the at least one predetermined or deduced conditions for the comparison at the first node.

13. The system of claim 10, further including:

at least one time dependent buffer comprising a variable length bit map;

wherein the processor further implements the first processing node for;

determining whether the decoded packets are wanted for further processing;

discarding unwanted decoded packets;

assigning wanted decoded packets to the at least one time dependent buffer for comparison to the at least one of the set of predetermined or deduced conditions;

forwarding wanted packets to the assigned at least one time dependent buffer for comparison to the at least one set of predetermined or deduced conditions.

14. The system of claim 11, wherein the processor further implements the first processing node for transmitting the wanted data packets to the second processing node for the step of processing.

15. The system of claim 10, wherein the processor further implements the second processing node for:

receiving the indication of detection of one of the at least one predetermined or deduced conditions;

assigning at least one predefined application associated with the indication;

processing the decoded data flow using the assigned at least one predefined application in simultaneous multithreaded parallel fashion to generate at least one control action for controlling the operation in the network at the same time the live-data flow is in active transmission between the endpoints in the network, wherein controlling comprises alerting or instructing a network provider to affect an event outcome before the event is finalized; and

controlling the operation of the network using the at least one control action.

16. The system of claim 10, wherein the processor further implements the second processing node for processing the portion of the decoded packets decoded data flow using relational language processing.

17. The system of claim 10, wherein the processor further implements the second processing node for generating an alert to another processor node in response to the at least one predetermined or deduced condition.

18. The system of claim 10, wherein the processor further implements the second processing node for processing at least a portion of the decoded packets of the live-data flow causing execution of the predetermined response and the deducing of dynamically created conditions responsive to historical data, the dynamically deduced conditions established as part of a learning process including statistical models and pattern recognition, operating within the second node with the option to provide the dynamically deduced conditions to the first processing node to update the deduced conditions.

19. The system of claim 10, wherein the processor further implements the second processing node for controlling the network by monitoring for an occurrence of network security threats within the live-data of the data flow and isolating any detected security threats within the live-data from the data flow of the network.

20. The system of claim 10, wherein the processor further implements the second processing node for monitoring the live-data of the data flow to create a model of subscriber movement and frequency of presence within each cell tower location, wherein each cell tower has planned and unplanned outages, further wherein as these outages occur, the second processor generating a roster of all subscribers currently within or approaching the cell tower, and generating a notification to be broadcast at least one of generally or selectively based on each subscribers communications preferences.

21. A system for monitoring live-data flow through a network, comprising:

a network interface for connecting to the network;

a processor coupled to the network interface;

a memory coupled to the processor, the memory storing a plurality of instructions for execution by the processor, the plurality of instructions including;

instructions for monitoring, at a first processing node, a mirrored live-data flow of the live-data flow passing through a selected point within the network in a non-intrusive manner that does not affect the live-data flow passing through the selected point, wherein the live-data flow comprises data that is in active transmission between endpoints in the network and prior to storage of the data in a database;

instructions for decoding, at the first processing node, each packet within the mirrored data flow according to each protocol associated with a packet, wherein packets have a plurality of protocols associated therewith are decoded in parallel with each other;

instructions for comparing, at the first processing node, each of the decoded packets to at least one of a set of predetermined or deduced conditions;

instructions for executing at least one of a predetermined or deduced response including an indication of occurrence of the at least one predetermined or deduced condition based upon detection of at least one of a predetermined or deduced condition within the decoded packets;

instructions for processing, at a second processing node, at least a portion of the decoded packets of the live-data flow causing execution of the at least one predetermined or deduced response to determine a manner for controlling an operation of the network at a same time the live-data flow is in active transmission between the endpoints in the network; and

instructions for controlling the operation of the network in response to the processing step, while events associated with the live-data flow are occurring within the network.

22. The system of claim 21, wherein the instructions for decoding further include:

instructions for determining whether the decoded packets are wanted for further processing;

instructions for discarding unwanted decoded packets;

instructions for assigning wanted decoded packets to at least one time dependent buffer for comparison to the at least one predetermined or deduced conditions, the time dependent buffer comprising a variable length bit map;

instructions for forwarding wanted packets to the assigned at least one time dependent buffer for comparison to the at least one of the set of predetermined or deduced conditions.

23. The system of claim 22 further including instructions for transmitting the wanted data packets to the second processing node for the processing of the portion of the decoded packets.

24. The system of claim 21, wherein the instructions for processing further comprises:

instructions for receiving the indication of detection of one of the predetermined or deduced conditions;

instructions for assigning at least one predefined application associated with the indication;

instructions for processing the decoded data flow using the assigned at least one predefined application in simultaneous multithreaded parallel fashion to generate at least one control action for controlling the operation in the network at the same time the live-data flow is in active transmission between the endpoints in the network, wherein controlling comprises alerting or instructing a network provider to affect an event outcome before the event is finalized; and

instructions for controlling the operation of the network using the at least one control action.

25. The system of claim 21, wherein the instructions for processing further comprises instructions for processing the portion of the decoded packets decoded data flow using relational language processing.

26. The system of claim 21, wherein the instructions for executing at least one predetermined or deduced response further comprises instructions for generating a notification to the network provider in response to the at least one predetermined or deduced condition.

27. The system of claim 21, wherein the instructions for executing the at least one predetermined or deduced response further comprises instructions for processing at least a portion of the decoded packets of the live-data flow causing execution of the predetermined response and the deducing of dynamically created conditions responsive to historical data, the dynamically deduced conditions established as part of a learning process including statistical models and pattern recognition, operating within the second node with the option to provide the dynamically deduced conditions to the first processing node to update the deduced conditions.

28. The system of claim 21, wherein the instructions for controlling further comprises instructions for controlling the network by monitoring for an occurrence of network security threats within the live-data of the data flow and isolating any detected security threats within the live-data from the data flow of the network, wherein controlling comprises alerting or instructing a network provider to affect an event outcome before the event is finalized.

29. The system of claim 21, wherein the instructions for controlling further comprises instructions for monitoring the live-data of the data flow to create a model of subscriber movement and frequency of presence within each cell tower location, wherein each cell tower has planned and unplanned outages, further wherein as these outages occur, the second processing generating a roster of all subscribers currently within or approaching the cell tower, and generating a notification to be broadcast at least one of generally or selectively based on each subscribers communications preferences.