Method and apparatus for automatically constructing application signatures

• US 7,620,807 B1
• Filed: 02/11/2005
• Issued: 11/17/2009
• Est. Priority Date: 02/11/2004
• Status: Expired due to Fees

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Spatscheck, Oliver, Sen, Subhabrata, Wang, Dongmei
• Primary Examiner(s): Zand; Kambiz
• Assistant Examiner(s): RAHIM, MONJUR
• Application Number:
  US11/056,896
• Time in Patent Office: 1,740 Days
• Field of Search: 706/26, 706/25, 706/16, 714/732, 704/257, 713/166, 713/200, 713/201, 713/152, 370/400
• US Class Current: 713/152
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/2441 relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers

Abstract

The present invention relates to a method and system for the automated construction of application signatures. In one example, an approach for automatically constructing accurate signatures for individual applications, with minimal human involvement or application domain knowledge, is provided. Given a training data set containing the application traffic, the Automated Construction of Application Signatures (ACAS) system uses a combination of statistical, information theoretic and combinatorial optimization techniques, to derive application-layer signatures from the payload of packets, e.g., IP packets. Evaluations with a range of applications demonstrate that the derived signatures are very accurate and scale to identifying a large number of flows in real time on high-speed links.

Claims

1. A method for generating a final signature for an application, comprising:

deriving an initial application signature using a model training data set;

testing said initial application signature with a single validation data set;

obtaining at least one cost minimizing parameter;

creating a plurality of trained application signatures using both said at least one cost minimizing parameter and a plurality of preclassified training data sets;

testing each of said plurality of trained application signatures using a plurality of preclassified validation data sets; and

determining said final signature from said plurality of trained application signatures.

2. The method of claim 1, wherein said deriving comprises:

identifying application layer features from data packets in said model training data set;

selecting relevant features from said application layer features; and

constructing said initial application signature from said selected relevant features.

3. The method of claim 2, wherein said selecting is conducted via mutual information.

4. The method of claim 2, wherein said application layer features comprise at least one of:

a bit feature, a word feature, or a Hypertext Transfer Protocol (HTTP) feature.

5. The method of claim 1, wherein said creating comprises:

identifying application layer features from data packets in each of said plurality of preclassified training data sets;

selecting relevant features from said application layer features; and

constructing one of said plurality of trained application signatures from said selected relevant features.

6. The method of claim 4, wherein said selecting is conducted via mutual information.

7. The method of claim 5, wherein said application layer features comprise at least one of:

a bit feature, a word feature, or a Hypertext Transfer Protocol (HTTP) feature.

8. The method of claim 1, wherein said final signature comprises one of said plurality of trained application signatures that minimizes a cost.

9. A system for generating a final signature for an application, comprising:

means for deriving an initial application signature using a model training data set;

means for testing said initial application signature with a single validation data set;

means for obtaining at least one cost minimizing parameter;

means for creating a plurality of trained application signatures using both said at least one cost minimizing parameter and a plurality of preclassified training data sets;

means for testing each of said plurality of trained application signatures using a plurality of preclassified validation data sets; and

means for determining said final signature from said plurality of trained application signatures.

10. The system of claim 9, wherein said means for deriving comprises:

means for identifying application layer features from data packets in said model training data set;

means for selecting relevant features from said application layer features; and

means for constructing said initial application signature from said selected relevant features.

11. The system of claim 10, wherein said means for selecting is conducted via mutual information.

12. The system of claim 9, wherein said means for creating comprises:

means for identifying application layer features from data packets in each of said plurality of preclassified training data sets;

means for selecting relevant features from said application layer features; and

means for constructing one of said plurality of trained application signatures from said selected relevant features.

13. The system of claim 12, wherein said means for selecting is conducted via mutual information.

14. The system of claim 9, wherein said final signature comprises one of said plurality of trained application signatures that minimizes a cost.

15. A computer readable medium having stored thereon instruction that, when executed by a processor, causing the processor to perform a method for generating a final signature using machine learning, comprising:

deriving an initial application signature using a model training data set;

testing said initial application signature with a single validation data set;

obtaining at least one cost minimizing parameter;

creating a plurality of trained application signatures using both said at least one cost minimizing parameter and a plurality of preclassified training data sets;

testing each of said plurality of trained application signatures using a plurality of preclassified validation data sets; and

determining said final signature from said plurality of trained application signatures.

16. The computer readable medium of claim 15, wherein said deriving comprises:

identifying application layer features from data packets in said model training data set;

selecting relevant features from said application layer features; and

constructing said initial application signature from said selected relevant features.

17. The computer readable medium of claim 16, wherein said selecting is conducted via mutual information.

18. The computer readable medium of claim 15, wherein said creating step comprises:

identifying application layer features from data packets in each of said plurality of preclassified training data sets;

selecting relevant features from said application layer features; and

constructing one of said plurality of trained application signatures from said selected relevant features.

19. The computer readable medium of claim 18, wherein said selecting is conducted via mutual information.

20. The computer readable medium of claim 15, wherein said final signature comprises one of said plurality of trained application signatures that minimizes a cost.