Systems and methods for extracting media from network traffic having unknown protocols

• US 10,547,523 B2
• Filed: 10/29/2015
• Issued: 01/28/2020
• Est. Priority Date: 06/08/2010
• Status: Active Grant

• Current Assignee: Cognyte Technologies Israel Ltd. (Cognyte Software Ltd.)
• Original Assignee: Verint Systems Limited (Verint Systems Incorporated)
• Inventors: Horovitz, Itsik
• Primary Examiner(s): Abelson, Ronald B
• Application Number:
  US14/926,675
• Publication Number:
  US 20160142273A1
• Time in Patent Office: 1,552 Days
• Field of Search: 709224
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/18 Protocol analysers
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data

Abstract

Methods and systems for analyzing network traffic. An analysis system receives network traffic, which complies with a certain protocol. The received network traffic carries a data item, which may be of value to an analyst. In order to access the data item in question, the analysis system automatically identifies the media type of the data item, by processing the network traffic without decoding the protocol. The analysis system identifies the media type irrespective of the protocol in order to avoid the computational complexity involved in decoding the protocol.

Claims

1. A method comprising:

receiving, in a computerized analysis system, a segment of network traffic that is exchanged between network users, wherein the segment of network traffic complies with an unknown protocol and carries part of a data item of a respective media type, wherein the unknown protocol is associated with a layer that is higher than layer 4 of the Open System Interconnection (OSI) reference model, wherein the computerized analysis system is not a designated participant in the network traffic;

automatically identifying the media type by processing the segment of network traffic as a sequence of bytes without decoding the unknown protocol and detecting in the sequence of bytes a characteristic that is indicative of the respective media type; and

extracting at least the part of the data item responsively to the identified media type, wherein extracting the data item comprises selecting a modality for presenting the data item responsively to the identified media type, and presenting the extracted data item to an operator using the selected modality, wherein automatically identifying the media type comprises one of identifying that the sequence of bytes comprises valid text or identifying in the network traffic a file type that is associated with the media type.

2. The method according to claim 1, wherein identifying the valid text comprises identifying a character set that is used for constructing the sequence of bytes.

3. The method according to claim 1, wherein identifying the valid text comprises verifying at least one criterion selected from a group of criteria consisting of:

the sequence is longer than a predefined minimal length;

at least a predefined portion of the bytes in the sequence represent letters;

the sequence comprises only bytes representing valid displayable characters; and

the sequence comprises at least one consecutive run of multiple alphanumerical characters.

4. The method according to claim 1, wherein identifying the valid text comprises assessing a statistical property of the bytes, and identifying that the text is valid responsively to the assessed statistical property.

5. The method according to claim 4, wherein assessing the statistical property comprises assessing an occurrence frequency of a given character in the sequence of the bytes.

6. The method according to claim 4, wherein assessing the statistical property comprises assessing a measure of randomness of the sequence of the bytes.

7. The method according to claim 4, wherein assessing the statistical property comprises comparing a first statistical distribution of the bytes in the sequence with a second statistical distribution of characters in a given language.

8. The method according to claim 1, wherein identifying the file type comprises detecting a byte pattern that is characteristic of the file type.

9. The method according to claim 1, wherein presenting the data item to the operator using the selected modality includes sounding out audio to the operator using a speaker or headset upon the media type of the data item being identified as audio.

10. An apparatus, comprising:

an interface configured to receive a segment of network traffic that is exchanged between network users, wherein the segment of network traffic complies with an unknown protocol and carries part of a data item of a respective media type, and wherein the unknown protocol is associated with a layer that is higher than layer 4 of the Open System Interconnection (OSI) reference model, wherein the apparatus is not a designated participant in the network traffic; and

a processor, which is configured to automatically identify the media type by processing the segment of network traffic as a sequence of bytes without decoding the unknown protocol, detect in the sequence of bytes a characteristic that is indicative of the respective media type, to extract at least the part of the data item responsively to the identified media type, and present the extracted data item to the operator using the selected modality, wherein the processor configured to automatically identify the media type comprises one of the processor conjured to identify that the sequence of bytes comprises valid text or the processor conjured to identify in the network traffic a file type that is associated with the media type.

11. The apparatus according to claim 10, wherein the processor is configured to identify the valid text by identifying a character set that is used for constructing the sequence of bytes.

12. The apparatus according to claim 10, wherein the processor is configured to identify the valid text by verify at least one criterion selected from a group of criteria consisting of:

the sequence is longer than a predefined minimal length;

at least a predefined portion of the bytes in the sequence represent letters;

the sequence comprises only bytes representing valid displayable characters; and

the sequence comprises at least one consecutive run of multiple alphanumerical characters.

13. The apparatus according to claim 10, wherein the processor is configured to sound out audio to the operator using a speaker or headset upon the media type of the data item being identified as audio by the processor.

14. A non-transitory computer-readable medium having stored thereon instructions that, when executed by a computerized analysis system, direct the analysis system to execute the process comprising the steps of:

receiving, in the computerized analysis system, a segment of network traffic that is exchanged between network users, wherein the segment of network traffic complies with an unknown protocol and carries part of a data item of a respective media type, and wherein the unknown protocol is associated with a layer that is higher than layer 4 of the Open System Interconnection (OSI) reference model, wherein the computerized analysis system is not a designated participant in the network traffic;

automatically identifying the media type by processing the segment of network traffic as a sequence of bytes without decoding the the unknown protocol and detecting in the sequence of bytes a characteristic that is indicative of the respective media type; and

extracting at least the part of the data item responsively to the identified media type, wherein extracting the data item comprises selecting a modality for presenting the data item responsively to the identified media type, and presenting the extracted data item to an operator using the selected modality, wherein automatically identifying the media type comprises one of identifying that the sequence of bytes comprises valid text or identifying in the network traffic a file type that is associated with the media type.

15. The non-transitory computer readable medium according to claim 14, wherein identifying the valid text comprises identifying a character set that is used for constructing the sequence of bytes.

16. The non-transitory computer readable medium according to claim 14, wherein identifying the valid text comprises verifying at least one criterion selected from a group of criteria consisting of:

the sequence is longer than a predefined minimal length;

at least a predefined portion of the bytes in the sequence represent letters;

the sequence comprises only bytes representing valid displayable characters; and

the sequence comprises at least one consecutive run of multiple alphanumerical characters.