Apparatus and method of analyzing internet activity

• US 5,787,253 A
• Filed: 05/28/1996
• Issued: 07/28/1998
• Est. Priority Date: 05/28/1996
• Status: Expired due to Fees

• Current Assignee: Silkstream Corporation
• Original Assignee: The AG Group
• Inventors: McCreery, Timothy David, Zabetian, Mahboud
• Primary Examiner(s): PEESO, THOMAS R
• Application Number:
  US08/654,347
• Time in Patent Office: 791 Days
• Field of Search: 395/200.11, 395/200.32, 395/200.61, 395/200.68, 395/200.75, 395/200.77, 370/354, 370/355, 370/245, 370/351, 370/379, 370/389, 370/392
• US Class Current: 709/231
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/0882 Utilisation of link capacity
• CPC Class Codes: H04L 43/18 Protocol analysers
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/161 Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

An internet activity analyzer includes a network interface controller, a packet capturing module, a packet analysis module, and a data management module. The network interface controller is connected to a transmission medium for a network segment and is arranged to receive the stream of data packets passing along the medium. The packet stream is filtered to remove undesired packet data and is stored in a raw packet data buffer. The packet data is decoded at the internet protocol layer to provide information such as timing and sequencing data regarding the exchange of packets between nodes and the packet data for exchanges between multiple nodes may be recompiled into concatenated raw transaction data which may be coherently stored in a raw transaction data buffer. An application level protocol translator translates the raw transaction data and stores the data in a translated transaction data buffer. The translated data provides high level information regarding the transactions between nodes which is used to monitor or compile statistics regarding network or internetwork activity. The data management module communicates with the packet capturing module and the packet analyzer and, particularly, the data in the raw packet, decoded packet, raw transaction, and translated transaction data buffers to provide real time and stored analytical information concerning internet activity.

Claims

1. An apparatus for analyzing internet activity, the apparatus comprising:

a packet capturing module, for accessing the packets traversing a network, the packets having source and destination addresses other than an address corresponding to the apparatus, and for filtering the packets to produce raw packet data, wherein the packet capturing module produces the raw packet data by retrieving a predetermined address, comparing the predetermined address to the internet protocol source address for a current packet, comparing the predetermined address to the internet protocol destination address for the current packet, and retaining the current packet where one of the internet protocol source and destination addresses for the current packet matches the predetermined address;

a packet analyzing module, in communication with the packet capturing module, for producing decoded packet data and for producing transaction data from the decoded packet data; and

a data management module, in communication with the packet capturing module and the packet analyzing module, for analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

2. An apparatus for analyzing internet activity, the apparatus comprising:

a packet capturing module, for accessing the packets traversing a network, the packets having source and destination addresses other than an address corresponding to the apparatus, and for filtering the packets to produce raw packet data, wherein the packet capturing module produces the raw packet data by retrieving a predetermined port address, comparing the predetermined port address to the transmission control protocol source port address for a current packet, comparing the predetermined port address to the transmission control protocol destination port address for the current packet, and retaining the current packet where one of the transmission control protocol source and destination port addresses for the current packet matches the predetermined port address;

a packet analyzing module, in communication with the packet capturing module, for producing decoded packet data and for producing transaction data from the decoded packet data; and

a data management module, in communication with the packet capturing module and the packet analyzing module, for analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

3. An apparatus for analyzing internet activity, the apparatus comprising:

a packet capturing module, for accessing the packets traversing a network, the packets having source and destination addresses other than an address corresponding to the apparatus, and for filtering the packets to produce raw packet data;

a packet analyzing module, in communication with the packet capturing module, for producing decoded packet data and for producing transaction data from the decoded packet data, the packet analyzing module comprising;

a packet decoder, for accessing the raw packet data and producing the decoded packet data; and

a decoded packet recompiler, in communication with the packet decoder, for accessing the decoded packet data, segregating the packets from the decoded packet data into separate transactions between nodes, sequencing the packets corresponding to each separate transaction, and concatenating the data in each separate transaction to produce the transaction data; and

a data management module, in communication with the packet capturing module and the packet analyzing module, for analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

4. The apparatus of claim 3, wherein the transaction data includes a first transaction and a second transaction, the first transaction comprising packets exchanged between a first node and a second node and the second transaction comprising packets exchanged between a third node and a fourth node.

5. An apparatus for analyzing internet activity, the apparatus comprising:

a packet capturing module, for accessing the packets traversing a network, the packets having source and destination addresses other than an address corresponding to the apparatus, and for filtering the packets to produce raw packet data;

a packet analyzing module, in communication with the packet capturing module, for producing decoded packet data, for producing transaction data from the decoded packet data, and for producing translated transaction data from the transaction data, the packet analyzing module comprising;

a packet decoder, for accessing the raw packet data and producing the decoded packet data;

a decoded packet recompiler, in communication with the packet decoder, for accessing the decoded packet data, segregating the packets from the decoded packet data into separate transactions between nodes, sequencing the packets corresponding to each separate transaction, and concatenating the data in each separate transaction to produce the transaction data; and

an application protocol translator, in communication with the decoded packet recompiler, for producing the translated transaction data by accessing the transaction data, scanning the transaction data for a field corresponding to a selected application protocol, determining a value associated with the field, and storing the field and the associated value; and

a data management module, in communication with the packet capturing module and the packet analyzing module, for analyzing at least one of the raw packet data, the decoded packet data, the transaction data, and the translated transaction data to provide an indication of internet usage.

6. The apparatus of claim 5, wherein the transaction data includes a first transaction and a second transaction, the first transaction comprising packets exchanged between a first node and a second node and the second transaction comprising packets exchanged between a third node and a fourth node.

7. The apparatus of claim 5, wherein the data management module produces a profile of a selected site by determining the number of transactions in which the selected site participates.

8. The apparatus of claim 7, wherein the data management module produces analytical data corresponding to selected transactions that the selected site participates in, the analytical data for each selected transaction including at least one of the browser type used to access the selected site, the amount of data transferred in the transaction, and the transaction time.

9. The apparatus of claim 7, wherein the data management module accesses the addresses for nodes transacting with the selected site, uses a reverse domain name service lookup to obtain information about the nodes, and uses the information about the nodes to produce the profile of the selected site.

10. For use with an internet activity analyzer capable of being coupled to a network transmission medium, a method of analyzing internet activity, the method comprising:

accessing the packets traversing the network, the packets having source and destination addresses other than an address corresponding to the internet activity analyzer;

filtering the packets to produce raw packet data by retrieving a predetermined address;

comparing the predetermined address to the internet protocol source address for a current packet;

comparing the predetermined address to the internet protocol destination address for the current packet; and

retaining the current packet where one of the internet protocol source and destination addresses for the current packet matches the predetermined address;

producing decoded packet data;

producing transaction data from the decoded packet data; and

analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

11. For use with an internet activity analyzer capable of being coupled to a network transmission medium, a method of analyzing internet activity, the method comprising:

accessing the packets traversing the network, the packets having source and destination addresses other than an address corresponding to the internet activity analyzer;

filtering the packets to produce raw packet data by retrieving a predetermined port address;

comparing the predetermined port address to the transmission control protocol source port address for a current packet;

comparing the predetermined port address to the transmission control protocol destination port address for the current packet; and

retaining the current packet where one of the transmission control protocol source and destination port addresses for the current packet matches the predetermined port address;

producing decoded packet data;

producing transaction data from the decoded packet data; and

analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

12. For use with an internet activity analyzer capable of being coupled to a network transmission medium, a method of analyzing internet activity, the method comprising:

accessing the packets traversing the network, the packets having source and destination addresses other than an address corresponding to the internet activity analyzer;

filtering the packets to produce raw packet data;

producing decoded packet data;

producing transaction data from the decoded packet data by accessing the decoded packet data;

segregating the packets from the decoded packet data into separate transactions between nodes;

sequencing the packets corresponding to each separate transaction; and

concatenating the data in each separate transaction to produce the transaction data; and

analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

13. The method of claim 12, wherein the transaction data includes a first transaction and a second transaction, the first transaction comprising packets exchanged between a first node and a second node and the second transaction comprising packets exchanged between a third node and a fourth node.

14. For use with an internet activity analyzer capable of being coupled to a network transmission medium, a method of analyzing internet activity, the method comprising:

accessing the packets traversing the network, the packets having source and destination addresses other than an address corresponding to the internet activity analyzer;

filtering the packets to produce raw packet data;

producing decoded packet data;

producing transaction data from the decoded packet data by accessing the decoded packet data;

segregating the packets from the decoded packet data into separate transactions between nodes;

sequencing the packets corresponding to each separate transaction; and

concatenating the data in each separate transaction to produce the transaction data; and

producing translated transaction data from the transaction data; and

analyzing at least one of the raw packet data, the decoded packet data, the transaction data, and the translated transaction data to provide an indication of internet usage.

15. The method of claim 14, wherein the step of producing translated transaction data comprises:

accessing the transaction data;

scanning the transaction data for a field corresponding to a selected application protocol;

determining a value associated with the field; and

retaining an association between the field and the determined value.

16. The method of claim 15, wherein the transaction data includes a first transaction and a second transaction, the first transaction comprising packets exchanged between a first node and a second node and the second transaction comprising packets exchanged between a third node and a fourth node.

17. The method of claim 16, further comprising:

producing a profile of a selected site by determining the number of transactions in which the selected site participates.

18. The method of claim 17, wherein analytical data corresponding to selected transactions that the selected site participates in is produced, the analytical data for each selected transaction including at least one of:

the browser type used to access the selected site, the amount of data transferred in the transaction, and the transaction time.

19. The method of claim 17, further comprising:

accessing the addresses for nodes transacting with the selected site;

using a reverse domain name service lookup to obtain information about the nodes; and

using the information about the nodes to produce the profile of the selected site.

20. An apparatus for analyzing internet activity, the apparatus comprising:

means for accessing the packets traversing the network, the packets having source and destination addresses other than an address corresponding to the internet activity analyzer;

means for filtering the packets to produce raw packet data, wherein the means for filtering the packets to produce raw packet data includes routines for retrieving a predetermined address;

comparing the predetermined address to the internet protocol source address for a current packet;

comparing the predetermined address to the internet protocol destination address for the current packet; and

retaining the current packet where one of the internet protocol source and destination addresses for the current packet matches the predetermined address ;

means for producing decoded packet data;

means for producing transaction data from the decoded packet data; and

means for analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

21. An apparatus for analyzing internet activity, the apparatus comprising:

means for accessing the packets traversing the network, the packets having source and destination addresses other than an address corresponding to the internet activity analyzer;

means for filtering the packets to produce raw packet data, wherein the means for filtering the packets to produce raw packet data includes routines for retrieving a predetermined port address;

comparing the predetermined port address to the transmission control protocol source port address for a current packet;

comparing the predetermined port address to the transmission control protocol destination port address for the current packet; and

retaining the current packet where one of the transmission control protocol source and destination port addresses for the current packet matches the predetermined port address;

means for producing decoded packet data;

means for producing transaction data from the decoded packet data; and

means for analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

22. An apparatus for analyzing internet activity, the apparatus comprising:

means for accessing the packets traversing the network, the packets having source and destination addresses other than an address corresponding to the internet activity analyzer;

means for filtering the packets to produce raw packet data;

means for producing decoded packet data;

means for producing transaction data from the decoded packet data, wherein the means for producing transaction data includes routines for accessing the decoded packet data;

segregating the packets from the decoded packet data into separate transactions between nodes;

sequencing the packets corresponding to each separate transaction; and

concatenating the data in each separate transaction to produce the transaction data; and

means for analyzing at least one of the raw packet data, the decoded packet data and the transaction data to provide an indication of internet usage.

23. The apparatus of claim 22, wherein the transaction data includes a first transaction and a second transaction, the first transaction comprising packets exchanged between a first node and a second node and the second transaction comprising packets exchanged between a third node and a fourth node.