System and method for conditional analysis of network traffic
First Claim
1. A method, comprising:
- receiving network traffic that carries content items for processing by an analytics system;
extracting a content item from the network traffic;
finding whether the content item is a duplicate of previous content that was already processed by the analytics system and cached;
when the content item is found to duplicate the previous content, retrieving and outputting a cached analytics outcome of the content item;
when the content item is found not to duplicate any previous content, causing the analytics system to produce the analytics outcome for the content item, and caching the analytics outcome, wherein extracting the content item comprises deriving a respective unique identifier for the content item, and wherein finding that the content item does not duplicate comprises validating that the unique identifier does not match any identifier in a cache memory that caches identifiers of previous content items that were processed by the analytics system; and
for a given content item, counting a number of matching, occurrences of the given content item, and caching the number of matching occurrences in the cache memory in association with the unique identifier of the given content item, for use by the analytics system, wherein caching the number of matching occurrences comprises deleting from the cache memory the given content identifier if the number of matching occurrences during a predefined duration is lower than a predefined threshold, wherein the number of matching occurrences is multiplied by a weight factor that is based on a processing time of the content item.
3 Assignments
0 Petitions
Accused Products
Abstract
Embodiments that are described herein provide improved methods and systems for analyzing network traffic. The disclosed embodiments enable an analytics system to perform complex processing to only new, first occurrences of received content, while refraining from processing duplicate instances of that content. In a typical embodiment, the analytics results regarding the first occurring content are reported and cached in association with the content. For any duplicate instance of the content, the analytics results are retrieved from the cache without re-processing of the duplicate content. When using the disclosed techniques, the system still processes all first occurring content but not duplicate instances of content that was previously received and processed. In the embodiments described herein, input data comprises communication packets exchanged in a communication network.
-
Citations
16 Claims
-
1. A method, comprising:
-
receiving network traffic that carries content items for processing by an analytics system; extracting a content item from the network traffic; finding whether the content item is a duplicate of previous content that was already processed by the analytics system and cached; when the content item is found to duplicate the previous content, retrieving and outputting a cached analytics outcome of the content item; when the content item is found not to duplicate any previous content, causing the analytics system to produce the analytics outcome for the content item, and caching the analytics outcome, wherein extracting the content item comprises deriving a respective unique identifier for the content item, and wherein finding that the content item does not duplicate comprises validating that the unique identifier does not match any identifier in a cache memory that caches identifiers of previous content items that were processed by the analytics system; and for a given content item, counting a number of matching, occurrences of the given content item, and caching the number of matching occurrences in the cache memory in association with the unique identifier of the given content item, for use by the analytics system, wherein caching the number of matching occurrences comprises deleting from the cache memory the given content identifier if the number of matching occurrences during a predefined duration is lower than a predefined threshold, wherein the number of matching occurrences is multiplied by a weight factor that is based on a processing time of the content item. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. An apparatus, comprising:
-
an input circuit, which is configured to receive network traffic that carries content items for processing by an analytics system; and a processor coupled to a memory, which is configured to extract a content item from the network traffic, to find whether the content item is a duplicate of a previous content that was already processed by the analytics system and cached, to retrieve and output a cached analytics outcome of the content item when the content item is found to duplicate the previous content, and, when the content item is found not to duplicate any previous content, to cause the analytics system to produce the analytics outcome for the content item and to cache the analytics outcome, wherein extracting the content item comprises the processor deriving a respective unique identifier for the content item, and wherein finding that the content item does not duplicate comprises the processor validating that the unique identifier does not match any identifier in a cache memory that caches identifies of previous content items that were processed by the analytics system; wherein for a given content item the processor is further configured to count a number of matching occurrences of the given content item, and cache the number of matching occurrences in the cache memory in association with the unique identifier of the given content item, for use by the analytics system, wherein caching the number of matching occurrences comprises the processor deleting from the cache memory the given content identifier if the number of matching occurrences during a predefined duration is lower than a predefined threshold, wherein the number of matching occurrences is multiplied by a weight factor that is based on a processing time of the content item.
-
Specification