Real-time content detection in ISP transmissions

• US 8,190,581 B2
• Filed: 12/03/2008
• Issued: 05/29/2012
• Est. Priority Date: 12/03/2008
• Status: Expired due to Fees

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Loman, James M., Gerber, Alexandre, Spatscheck, Oliver, Todimala, Ajay
• Primary Examiner(s): Ali, Mohammad
• Assistant Examiner(s): HOCKER, JOHN P
• Application Number:
  US12/315,380
• Publication Number:
  US 20100138543A1
• Time in Patent Office: 1,273 Days
• Field of Search: 707/687, 707/705, 707/685
• US Class Current: 707/685
• CPC Class Codes: G06F 16/9014 hash tables
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: H04L 2463/103 applying security measure for protecting copy right protecting software against unauthorised usage in a vending or licensing environment, e.g. protection the software providers copyright G06F21/10; data processing systems or methods, specially adapted for payment schemes, architectures or protocols G06Q20/00; secrecy systems or subscription systems H04N7/16
• CPC Class Codes: H04L 41/0631 using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
• CPC Class Codes: H04L 43/026 using flow identification
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/30 Profiles

Abstract

A method and system for detecting the transmission of preidentified content, such as copyrighted material, over an Internet Service Provider (ISP) network. A set of rules is provided to identify one or more traffic flow profiles of data streams transmitting preidentified content. Preferably the rules are adaptively created through analysis of actual ISP data in conjunction with data suggesting an initial set of profile characteristics. The rules are applied to data streams being transmitted in the ISP network, so that data streams fitting one or more of the profiles are identified. A database contains, e.g., as digital signatures or fingerprints, one or more items of content whose transmission is sought to be detected. Data streams identified as matching a profile are analyzed to determine if their content matches an item of content in the database, and if so, an action is taken which may include interrupting the transmission, suspending an ISP account, or reporting the transmission. An ISP with a system performing this method may offer services to content providers, and a plurality of ISPs may jointly use a single database of preidentified content to be compared to each ISP'"'"'s identified data streams.

Claims

1. A method for operating network management apparatus to detect whether one or more items of preidentified content are present in a network traffic flow being transmitted over an Internet Service Provider (ISP) network, the network traffic flow including data traffic flows potentially carrying the preidentified content intermixed together with data traffic flows carrying other content that is to remain private, all the data in the network data traffic flows being in the form of packets having both a layer with content-free network traffic information and a layer with content information, comprising:

providing the network management apparatus with access to the network traffic flow being transmitted over the ISP network;

providing the network management apparatus with one or more profile identification rules based solely on packet network layer traffic information to identify one or more data traffic flows that correlate with the preidentified content;

applying the one or more profile identification rules to the network traffic flow as it is being transmitted over the ISP network to select for further analysis those data traffic flows in the network traffic flow that have network layer information that satisfies one or more of the network layer profile identification rules;

storing in a database apparatus the one or more items of preidentified content whose presence in the network traffic flow being transmitted over the ISP network is to be detected;

after selecting a data traffic flow satisfying the one or more network layer profile identification rules, further analyzing the selected data traffic flow by comparing the content of the selected data flow with the preidentified content stored in the database apparatus to determine if it matches an item of preidentified content in the database apparatus; and

if the content of the selected data traffic flow is a match with an item of preidentified content in the database apparatus, taking an action in response,

wherein providing the one or more profile identification rules to identify one or more data traffic flows that correlate with the preidentified content comprises adaptively creating the one or more profile identification rules by providing an initial set of network layer profile characteristics, processing data regarding the traffic flows within the ISP network by using the initial set of profile characteristics to determine an initial correlation with preidentified content, and adjusting the set of profile characteristics to improve their correlation with the preidentified content.

2. The method claimed in claim 1 wherein the step of processing data is performed with an online data analyzer.

3. The method claimed in claim 1 wherein the network layer profile identification rules measure network layer traffic information parameters selected from source of data, destination of data, duration of data transmission from a source, frequency of data transmission from a source, and file type of data transmitted.

4. The method claimed in claim 1 wherein applying the network layer profile identification rules to data streams being transmitted in the ISP network comprises applying the rules in an active in line deep packet inspection device embedded within an ISP network element.

5. The method claimed in claim 1 wherein storing one or more items of preidentified content in a database comprises storing characteristics of the items of preidentified content in a digital library.

6. The method claimed in claim 5 wherein the stored characteristics of the preidentified content are digital fingerprints or computed hash values of the preidentified content.

7. The method claimed in claim 6 wherein comparing the content of the selected data traffic flow with the stored preidentified content comprises comparing digital fingerprints or hash values of the content of the selected data traffic flow with the digital fingerprints or hash values of an item of preidentified content stored in the database apparatus.

8. The method claimed in claim 1 wherein taking an action comprises terminating a selected data traffic flow transmission.

9. The method claimed in claim 1 wherein taking an action comprises suspending an account of an ISP customer to whom a selected data traffic flow transmission is directed.

10. The method claimed in claim 1 wherein taking an action comprises reporting the existence of the matching data traffic flow transmission to an interested party.

11. The method claimed in claim 1 wherein the database apparatus and stored preidentified content are shared by a plurality of ISPs.

12. A network management system for detecting whether one or more items of preidentified content are present in a network traffic flow being transmitted over an Internet Service Provider (ISP) network, the network traffic flow including data traffic flows potentially carrying the preidentified content intermixed together with data traffic flows carrying other content that is to remain private, all the data in the network data traffic flows being in the form of packets having both a layer with content-free network traffic information and a layer with content information, comprising:

means providing one or more network layer profile identification rules based solely on packet network layer traffic information to identify one or more data traffic flows that correlate with the preidentified content;

means for applying the one or more network layer profile identification rules to the network traffic flow as it is being transmitted over the ISP network to select for further analysis those data traffic flows in the network traffic flow that have network layer information that satisfies one or more of the profile identification rules;

a database apparatus for storing one or more items of preidentified content whose presence in the network traffic flow being transmitted over the ISP network is to be detected;

means, after selecting a data traffic flow satisfying the one or more network layer profile identification rules, for further analyzing the selected data flow by comparing the content of the selected data flow with the preidentified content stored in the database apparatus to determine if it matches an item of preidentified content in the database apparatus; and

means for taking an action in response if there is a positive match between the content of the selected data traffic flow and an item of preidentified content in the database apparatus,

wherein the means providing one or more network layer profile identification rules to identify one or more data flows that correlate with the preidentified content comprises means for adaptively creating the one or more network layer profile identification rules by providing an initial set of network layer profile characteristics, processing data regarding the traffic flows within the ISP network by using the initial set of profile characteristics to determine an initial correlation with preidentified content, and adjusting the set of profile characteristics to improve their correlation with the preidentified content.

13. The system claimed in claim 12 wherein the means for processing data is an online data analyzer.

14. The system claimed in claim 12 wherein the network layer profile identification rules measure network layer information parameters selected from source of data, destination of data, duration of data transmission from a source, frequency of data transmission from a source, and file type of data transmitted.

15. The system claimed in claim 12 wherein the means for applying the network layer profile identification rules to data streams being transmitted in the ISP network comprises means performing an active in-line deep packet inspection embedded within an ISP network element.

16. The system claimed in claim 12 wherein the database apparatus storing one or more items of preidentified content stores characteristics of the items of preidentified content in a digital library.

17. The system claimed in claim 16 wherein the stored characteristics of the preidentified content are digital fingerprints or computed hash values of the preidentified content.

18. The system claimed in claim 17 wherein the means for comparing the content of the selected data traffic flow with the stored preidentified content comprises means for comparing digital fingerprints or hash values of the content of the selected data traffic flow with digital fingerprints or hash values of an item of preidentified content stored in the database apparatus.

19. The system claimed in claim 12 wherein the action taken comprises terminating identified selected data traffic flow transmission.

20. The system claimed in claim 12 wherein the action taken comprises suspending an account of an ISP customer to whom the selected data traffic flow transmission is directed.

21. The system claimed in claim 12 wherein the action taken comprises reporting the existence of the matching data traffic flow transmission to an interested party.

22. The system claimed in claim 12 wherein the database apparatus and stored preidentified content are shared by a plurality of ISPs.