Systems and methods for extracting media from network traffic having unknown protocols

• US 9,197,523 B2
• Filed: 03/24/2014
• Issued: 11/24/2015
• Est. Priority Date: 06/08/2010
• Status: Active Grant

• Current Assignee: Cognyte Technologies Israel Ltd. (Cognyte Software Ltd.)
• Original Assignee: Verint Systems Limited (Verint Systems Incorporated)
• Inventors: Horovitz, Itsik
• Primary Examiner(s): Abelson, Ronald B
• Application Number:
  US14/223,653
• Publication Number:
  US 20140289401A1
• Time in Patent Office: 610 Days
• Field of Search: 370/241, 370/242, 370/395.5, 370/465, 370/466, 707/755
• US Class Current: 1/1
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/18 Protocol analysers
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data

Abstract

Methods and systems for analyzing network traffic. An analysis system receives network traffic, which complies with a certain protocol. The received network traffic carries a data item, which may be of value to an analyst. In order to access the data item in question, the analysis system automatically identifies the media type of the data item, by processing the network traffic irrespective of the protocol. The analysis system identifies the media type irrespective of the protocol in order to avoid the computational complexity involved in decoding the protocol.

Claims

1. A method, comprising:

receiving in a computerized analysis system network traffic, which complies with a protocol, wherein the protocol is not decodable by the analysis system, and which carries a data item of a respective media type;

automatically identifying the media type by processing the network traffic without decoding the protocol; and

extracting at least part of the data item responsively to the identified media type.

2. The method according to claim 1, wherein the data item comprises a sequence of bytes, and wherein identifying the media type comprises identifying that the sequence of bytes comprises valid text.

3. The method according to claim 2, wherein identifying the valid text comprises identifying a character set that is used for constructing the sequence of bytes.

4. The method according to claim 2, wherein identifying the valid text comprises verifying at least one criterion selected from a group of criteria consisting of:

the sequence is longer than a predefined minimal length;

at least a predefined portion of the bytes in the sequence represent letters;

the sequence comprises only bytes representing valid displayable characters; and

the sequence comprises at least one consecutive run of multiple alphanumerical characters.

5. The method according to claim 2, wherein identifying the valid text comprises assessing a statistical property of the bytes, and identifying that the text is valid responsively to the assessed statistical property.

6. The method according to claim 5, wherein assessing the statistical property comprises assessing an occurrence frequency of a given character in the sequence of the bytes.

7. The method according to claim 5, wherein assessing the statistical property comprises assessing a measure of randomness of the sequence of the bytes.

8. The method according to claim 5, wherein assessing the statistical property comprises comparing a first statistical distribution of the bytes in the sequence with a second statistical distribution of characters in a given language.

9. The method according to claim 1, wherein identifying the media type comprises identifying in the network traffic a file type that is associated with the media type.

10. The method according to claim 9, wherein identifying the file type comprises detecting a byte pattern that is characteristic of the file type.

11. The method according to claim 1, wherein extracting the data item comprises selecting a modality for presenting the data item responsively to the identified media type, and presenting the data item to an operator using the selected modality.

12. An apparatus, comprising:

an interface configured to receive network traffic, which complies with a protocol, wherein the protocol is not decodable by the analysis system, and which carries a data item of a respective media type; and

a processor, which is configured to automatically identify the media type by processing the network traffic without decoding the protocol, and to extract at least part of the data item responsively to the identified media type.

13. The apparatus according to claim 12, wherein the data item comprises a sequence of bytes, and wherein the processor is configured to identify that the sequence of bytes comprises valid text.

14. The apparatus according to claim 13, wherein the processor is configured to identify the valid text by identifying a character set that is used for constructing the sequence of bytes.

15. The apparatus according to claim 13, wherein the processor is configured to identify the valid text by verify at least one criterion selected from a group of criteria consisting of:

the sequence is longer than a predefined minimal length;

at least a predefined portion of the bytes in the sequence represent letters;

the sequence comprises only bytes representing valid displayable characters; and

the sequence comprises at least one consecutive run of multiple alphanumerical characters.

16. A non-transitory computer-readable medium having stored thereon instructions that, when executed by a computerized analysis system, direct the analysis system to execute the process comprising the steps of:

receiving in the computerized analysis system network traffic, which complies with a protocol, wherein the protocol is not decodable by the analysis system, and which carries a data item of a respective media type;

automatically identifying the media type by processing the network traffic without decoding the protocol; and

extracting at least part of the data item responsively to the identified media type.

17. The non-transitory computer readable medium according to claim 16, wherein the data item comprises a sequence of bytes, and wherein identifying the media type comprises identifying that the sequence of bytes comprises valid text.

18. The non-transitory computer readable medium according to claim 17, wherein identifying the valid text comprises identifying a character set that is used for constructing the sequence of bytes.

19. The non-transitory computer readable medium according to claim 17, wherein identifying the valid text comprises verifying at least one criterion selected from a group of criteria consisting of:

the sequence is longer than a predefined minimal length;

at least a predefined portion of the bytes in the sequence represent letters;

the sequence comprises only bytes representing valid displayable characters; and

the sequence comprises at least one consecutive run of multiple alphanumerical characters.