Object classification in a capture system

• US 20050289181A1
• Filed: 06/23/2004
• Published: 12/29/2005
• Est. Priority Date: 06/23/2004
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Deninger, William, de la Iglesia, Erik
• Granted Patent: US 7,962,591 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/564 Enhancement of application control based on intercepted application data
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

Objects can be extracted from data flows captured by a capture device. Each captured object can then be classified according to content. In one embodiment, the present invention includes determining whether a captured object is binary or textual in nature, and classifying the captured object as one of a plurality of textual content types based tokens found in the captured object if the captured object is determined to be textual in nature.

Claims

1. A method for classifying an object according to content comprising:

determining whether the object is binary or textual in nature; and

classifying the object as one of a plurality of textual content types based tokens found in the object if the object is determined to be textual in nature.

2. The method of claim 1, further comprising classifying the object as an encrypted document based on a statistical characteristic of the object if the object is determined to be binary in nature.

3. The method of claim 2, wherein the statistical characteristic of the object comprises a byte distribution of the object.

4. The method of claim 1, wherein classifying the object comprises generating a list of known tokens found in the object, each token being associated with a textual content type, calculating the probability of the object being each of the plurality of textual content types, and classifying the object as the textual content type with the highest probability.

5. The method of claim 4, wherein calculating the probability of the object being each of the plurality of textual content types comprises performing Bayesian statistics on the list of known tokens found in the object.

6. The method of claim 1, wherein the plurality of textual content types include Englishtext, Frenchtext, Germantext, Spanishtext, Japanesetext, Chinesetext, Koreantext, Russiantext, Basic_Source, C++_Source, C_Source, Java_Source, FORTRAN_Source, Verilog_Source, VHDL_Source, Assembly_Source, Pascal_Source, Cobol_Source, Ada_Source, Lisp_Source, Perl_Source, XQuery_Source, Hypertext Markup Language, Cascaded Style Sheets, JavaScript, DXF, Spice, Gerber, Mathematica, Matlab, AllegroPCB, ViewLogic, TangoPCAD, BSDL, C_Shell, K_Shell, Bash_Shell, Bourne_Shell, FTP, Telnet, MSExchange, POP3, RFC822, CVS, CMS, SQL, RTSP, MIME, PDF, PS, and Stockdata.

7. The method of claim 1, further comprising attempting to classify the object by searching for a known binary signature associated with a content type in the object.

8. The method of claim 1, wherein the object being textual in nature comprises the object containing ASCII text.

9. A method comprising:

intercepting a flow;

classifying the flow according to transmission protocol;

extracting one or more objects from the classified flow using a protocol handler corresponding with the transmission protocol; and

classifying the one or more objects based on content type by statistically analysing the object.

10. The method of claim 9, wherein classifying the flow according to protocol comprises not recognizing a transmission protocol carrying the flow and classifying the flow as unknown;

and wherein extracting one or more object comprises using a default protocol handler to extract the one or more objects from the unknown flow.

11. The method of claim 9, wherein the unknown protocol handler classifies the unknown flow as one object.

12. The method of claim 9, wherein statistically analysing the object comprises determining a byte distribution.

13. An apparatus comprising:

an object statistics module to determining whether an object is binary or textual in nature;

a token database to store a plurality of tokens, each token being associated with a textual content type; and

a token analyser to classify the object as one of the plurality of textual content types by accessing the token database if the object is determined to be textual in nature.

14. The apparatus of claim 13, wherein the object statistics module is configured to classify the object as an encrypted document based on a statistical characteristic of the object if the object is binary in nature.

15. The apparatus of claim 14, wherein the statistical characteristic of the object comprises a byte distribution of the object.

16. The apparatus of claim 13, the token analyser is configured to classify the object by generating a list of known tokens found in the object, each token being associated with a textual content type, calculating the probability of the object being each of the plurality of textual content types, and classifying the object as the textual content type with the highest probability.

17. The apparatus of claim 16, wherein the token analyser is configured to calculate the probability of the object being each of the plurality of textual content types by performing Bayesian statistics on the list of known tokens found in the object.

18. The apparatus of claim 13, wherein the plurality of textual content types include Englishtext, Frenchtext, Germantext, Spanishtext, Japanesetext, Chinesetext, Koreantext, Russiantext, Basic_Source, C++_Source, C_Source, Java_Source, FORTRAN_Source, Verilog_Source, VHDL_Source, Assembly_Source, Pascal_Source, Cobol_Source, Ada_Source, Lisp_Source, Perl_Source, XQuery_Source, Hypertext Markup Language, Cascaded Style Sheets, JavaScript, DXF, Spice, Gerber, Mathematica, Matlab, AllegroPCB, ViewLogic, TangoPCAD, BSDL, C_Shell, K_Shell, Bash_Shell, Bourne_Shell, FTP, Telnet, MSExchange, POP3, RFC822, CVS, CMS, SQL, RTSP, MIME, PDF, PS, and Stockdata.

19. The apparatus of claim 13, further comprising a binary signature module to classify the object by searching for a known binary signature associated with a content type in the object.

20. A machine-readable medium having stored thereon data representing instructions that, when executed by a processor of a capture device, cause the processor to perform operations comprising:

determining whether a captured object is binary or textual in nature; and

classifying the captured object as one of a plurality of textual content types based tokens found in the captured object if the captured object is determined to be textual in nature.

21. The machine-readable medium of claim 20, wherein the instructions further cause the processor to classify the captured object as an encrypted document based on a statistical characteristic of the captured object if the captured object is determined to be binary in nature.

22. The machine-readable medium of claim 21, wherein the statistical characteristic of the captured object comprises a byte distribution of the captured object.

23. The machine-readable medium of claim 20, wherein classifying the captured object comprises generating a list of known tokens found in the captured object, each token being associated with a textual content type, calculating the probability of the captured object being each of the plurality of textual content types, and classifying the captured object as the textual content type with the highest probability.

24. The machine-readable medium of claim 23, wherein calculating the probability of the captured object being each of the plurality of textual content types comprises performing Bayesian statistics on the list of known tokens found in the captured object.

25. The machine-readable medium of claim 20, wherein the plurality of textual content types include Englishtext, Frenchtext, Germantext, Spanishtext, Japanesetext, Chinesetext, Koreantext, Russiantext, Basic_Source, C++_Source, C_Source, Java_Source, FORTRAN_Source, Verilog_Source, VHDL_Source, Assembly_Source, Pascal_Source, Cobol_Source, Ada_Source, Lisp_Source, Perl_Source, XQuery_Source, Hypertext Markup Language, Cascaded Style Sheets, JavaScript, DXF, Spice, Gerber, Mathematica, Matlab, AllegroPCB, ViewLogic, TangoPCAD, BSDL, C_Shell, K_Shell, Bash_Shell, Bourne_Shell, FTP, Telnet, MSExchange, POP3, RFC822, CVS, CMS, SQL, RTSP, MIME, PDF, PS, and Stockdata.

26. The machine-readable medium of claim 20, wherein the instruction further cause the processor to attempt to classify the captured object by searching for a known binary signature associated with a content type in the captured object.

27. The machine-readable medium of claim 20, wherein the captured object being textual in nature comprises the captured object containing ASCII text.

28. A machine-readable medium having stored thereon data representing instructions that, when executed by a processor of a capture device, cause the processor to perform operations comprising:

intercepting a flow;

classifying the flow according to transmission protocol;

extracting one or more objects from the classified flow using a protocol handler corresponding with the transmission protocol; and

classifying the one or more objects based on content type by statistically analysing the object.

29. The machine-readable medium of claim 28, wherein classifying the flow according to protocol comprises not recognizing a transmission protocol carrying the flow and classifying the flow as unknown;

and wherein extracting one or more object comprises using a default protocol handler to extract the one or more objects from the unknown flow.

30. The machine-readable medium of claim 28, wherein the unknown protocol handler classifies the unknown flow as one object.

31. The machine-readable medium of claim 28, wherein statistically analysing the object comprises determining a byte distribution.