System and method for real-time analysis of network traffic

• US 9,529,621 B2
• Filed: 06/13/2016
• Issued: 12/27/2016
• Est. Priority Date: 09/13/2013
• Status: Expired due to Fees

• Current Assignee: Network Kinetix, Llc
• Original Assignee: Network Kinetix, Llc
• Inventors: Richards, Peter, Richards, Carissa
• Primary Examiner(s): KATSIKIS, KOSTAS J
• Application Number:
  US15/180,496
• Publication Number:
  US 20160299776A1
• Time in Patent Office: 197 Days
• Field of Search: 709/224
• US Class Current: 1/1
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5051 Service on demand, e.g. definition and deployment of services in real time
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/0894 Packet rate
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04M 15/47 Fraud detection or prevention means
• CPC Class Codes: H04M 15/8214 Data or packet based
• CPC Class Codes: H04M 15/8228 Session based
• CPC Class Codes: H04M 3/2218 Call detail recording
• CPC Class Codes: H04W 12/128 Anti-malware arrangements, e.g. protection against SMS fraud or mobile malware

Abstract

A system for monitoring live-data flow through a network includes a processor implementing a first processing node including an ingestor virtual machine (ingestor VM) for monitoring a mirrored live-data flow of the live-data flow passing through a selected point within the network in a non-intrusive manner that does not affect the live-data flow of at least one live data flow passing through the selected point. The ingestor VM further decodes each packet within the mirrored data flow according to each protocol associated with a packet and manages processes occurring within and between the first processing node and a second processing node. A time dependent buffer virtual machine (TDB VM) allocates a time dependent buffer (TDB) within the memory for executing the processes performed within and between the first processing node and a second processing node, and releasing the allocated TDB after completion of the processes. A governor virtual machine (governor VM) allocates memory resources within the memory between the first processing node and the second processing node for the processes performed within and between the first processing node and a second processing node. A grid virtual machine (grid VM) controls communications within and between the first processing nodes and between the first processing node and the second processing node.

Claims

1. A system for monitoring live-data flow through a network, comprising:

a server communicating with the network;

a memory associated with the server;

a processor within the server the processor implementing a first processing node and a second processing node, the first processing node including;

an ingestor virtual machine (ingestor VM) for configuring the processor to monitor a mirrored live-data flow of at least one live-data flow passing through a selected point within the network in a non-intrusive manner that does not affect the live-data flow passing through the selected point, wherein the live-data flow comprises data that is in active transmission between endpoints in the network and prior to storage of the data in a database, the ingestor VM decoding each packet within the mirrored data flow according to each protocol associated with a packet, wherein packets have a plurality of protocols associated therewith are decoded in parallel with each other, the ingestor VM further managing processes occurring within and between the first processing node and the second processing node and controlling the operation of the network;

a time dependent buffer virtual machine (TDB VM) for allocating a time dependent buffer (TDB) within the memory for executing the processes occurring within and between the first processing node and the second processing node and releasing the allocated TDB after completion of the processes;

a governor virtual machine (governor VM) for allocating memory resources within the memory between the first processing node and the second processing node for the processes occurring within and between the first processing node and the second processing node; and

a grid virtual machine (grid VM) for controlling communications within the first processing node and between the first processing node and the second processing node.

2. The system of claim 1, wherein the TDB VM further configures the processor to allocate multiple time dependent buffers to enable multiple functions of disparate timing to be scheduled and executed to provide optimum memory availability to the functions.

3. The system of claim 2, wherein the functions include at least one of decoding, comparing, executing predetermined or deduced responses, forwarding from the first processing node to the second processing node and controlling operation of the network.

4. The system of claim 2, wherein the memory resources are made available to functions in both the first and the second processing nodes.

5. The system of claim 2, wherein the TDB VM further configures the processor to make each available memory workspace available to the functions.

6. The system of claim 2, wherein a memory workspace may be made available to a specific function or shared in parallel by the multiple functions requiring access to the memory workspace.

7. The system of claim 6, where a single live data flow may be decoded in the memory workspace and accessed for processing by a plurality of applications in the second processing node, with some applications executing predetermined or deduced responses on the data flow.

8. The system of claim 2, wherein the TDB VM configures the processor to release assigned memory workspace after completion of the function assigned to the memory workspace to enable reassignment of the memory workspace to a second function.

9. The system of claim 1, wherein the governor VM configures the processor to monitor and maintain predetermined performance levels of memory usage and memory space available to all virtual machines and tasks within the first and second processing nodes.

10. The system of claim 1, wherein the governor VM configures the processor to ensure memory usage does not exceed a predetermined threshold responsive to measured time periods for completing tasks within and between the first and second processing nodes.

11. The system of claim 1, wherein the measured time periods include time periods for at least one of decoding, comparing, executing predetermined or deduced responses, forwarding from the first processing node to the second processing node and controlling operation of the network.

12. The system of claim 1, wherein the governor VM configures the processor to perform dynamic performance balancing of core utilization and memory resources when executing processes approach predetermined thresholds.

13. The system of claim 1, wherein the governor VM configures the processor to reallocate system priorities and memory resources within the first processing node and within and between the first processing node and the second processing node.

14. The system of claim 13, wherein the reallocation of system priorities and memory resources can occur in at least one of an operating system, core, node, and system levels.

15. The system of claim 13, wherein the governor VM reallocates resources responsive to a threshold level;

a specific function operating in a specific core in the first processing node being reallocated to an available same specific core in a neighboring first processing node.

16. The system of claim 13, wherein the governor VM reallocates resources responsive to a threshold level and reallocates a specific function operating in a core in the first processing node to any available core in a neighboring first processing node.

17. The system of claim 1, wherein the grid VM configures the processor to maintain a map of operations, connectivity, and performance at a node level and at an inter-node level.

18. The system of claim 1, wherein the grid VM configures the processor to communicate dynamic conditions and required actions within and between the first processing node and the second processing node responsive to commands from the governor VM and the TDB VM.

19. The system of claim 1, wherein the TDB VM, grid VM and governor VM enable an operational analysis for the functioning of the system under current and historical conditions for optimal performance and operations.

20. A method for monitoring live-data flow through a network, comprising:

monitoring, at a first processing node, a mirrored live-data flow of at least one live-data flow passing through a selected point within the network in a non-intrusive manner that does not affect the live-data flow passing through the selected point, wherein the live-data flow comprises data that is in active transmission between endpoints in the network and prior to storage of the data in a database;

decoding, at the first processing node, each packet within the mirrored data flow according to each protocol associated with a packet, wherein packets have a plurality of protocols associated therewith are decoded in parallel with each other;

allocating, at the first processing node, memory resources within a memory between the first processing node and a second processing node for the monitoring and the decoding of the live-data flow passing through the selected point and managing tasks occurring within and between the first processing node and the second processing node, and controlling operation of the network;

controlling, at the first processing node, communications within the first processing node and within and between the first processing node and the second processing node to enable execution of the processes occurring with respect to the first processing node and the second processing node;

allocating, at the first processing node, a time dependent buffer (TDB) within the memory for performing the processes occurring with respect to the first processing node and the second processing node; and

releasing, at the first processing node, the allocated TDB after performing the processes occurring with respect to the first processing node and the second processing node.

21. The method of claim 20, wherein the step of allocating the TDB further comprises allocating multiple time dependent buffers to enable multiple functions of disparate timing to be scheduled and executed to provide optimum memory availability to the functions.

22. The method of claim 20, wherein functions include at least one of decoding, comparing, executing predetermined or deduced responses, forwarding from the first processing node to the second processing node and controlling operation of the network.

23. The method of claim 20, wherein memory workspaces are made available to functions in both the first and the second processing nodes.

24. The method of claim 20, wherein the step of allocating the TDB further comprises making each available memory workspace available to the functions.

25. The method of claim 20, wherein a memory workspace may be made available to a specific function or shared in parallel by multiple functions requiring access to the memory workspace.

26. The method of claim 20, wherein a single live data flow may be decoded in a memory workspace and accessed for comparison or analysis by a plurality of applications in the second processing node, with some applications executing predetermined or deduced responses on the data flow.

27. The method of claim 20, wherein the step of allocating the TBD further comprises releasing assigned memory workspace after completion of the function assigned to the memory workspace to enable reassignment of the memory workspace to a second function.

28. The method of claim 20, further including the steps of monitoring and maintaining predetermined performance levels of memory usage and making memory space available to all virtual machines and tasks within the first and second processing node.

29. The method of claim 20, wherein the step of allocating memory resources further comprises measuring time periods for completing tasks within and between the first and second processing node to ensure memory usage does not exceed a predetermined threshold.

30. The method of claim 29, wherein the measured time periods include time periods for at least one of decoding, comparing, executing predetermined or deduced responses, forwarding from the first processing node to the second processing node and controlling operation of the network.

31. The method of claim 20, wherein the step of allocating memory resources further comprises performing dynamic performance balancing of core utilization and memory resources when executing processes approach predetermined thresholds.

32. The method of claim 20, wherein the step of allocating memory resources further comprises reallocating system priorities and memory resources within the first processing node and between the first processing node and the second processing node.

33. The method of claim 32, wherein the reallocation of the system priorities and the memory resources occur at an operating system, core, node, and system levels.

34. The method of claim 32, wherein the reallocation of resources is responsive to a threshold level and further comprises reallocating a specific function operating in a specific core in the first processing node being reallocated to an available same specific core in a neighboring first processing node.

35. The method of claim 32 wherein the reallocation of resources is responsive to a threshold level and further comprises reallocating a specific function operating in a core in the first processing node being reallocated to any available core in a neighboring first processing node.

36. The method of claim 20, wherein the step of controlling communications further comprises maintaining a map of operations, connectivity and performance at a node level and at an inter-node level.

37. The method of claim 20, wherein the step of controlling communications further comprises communicating dynamic conditions and required actions within and between the first processing node and the second processing node responsive to commands from a governor VM and a TDB VM.

38. The method of claim 20, wherein the TDB VM, grid VM and governor VM enable an operational analysis for functioning of the system under current and historical conditions for optimal performance and operations.