System and method for monitoring and analyzing internet traffic

• US 9,185,016 B2
• Filed: 08/14/2013
• Issued: 11/10/2015
• Est. Priority Date: 10/04/1999
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Muret, Paul N., Moon, Hui Sok
• Primary Examiner(s): LU, KUEN S
• Application Number:
  US13/966,433
• Publication Number:
  US 20140025814A1
• Time in Patent Office: 818 Days
• US Class Current: 1/1
• CPC Class Codes: A61K 38/00 Medicinal preparations containing peptides peptides containing beta-lactam rings A61K31/00; cyclic dipeptides not having in their molecule any other peptide link than those which form their ring, e.g. piperazine-2,5-diones, A61K31/00; ergot alkaloids of the cyclic peptide type A61K31/48; containing macromolecular compounds having statistically distributed amino acid units A61K31/74; medicinal preparations containing antigens or antibodies A61K39/00; medicinal preparations characterised by the non-active ingredients, e.g. peptides as drug carriers, A61K47/00
• CPC Class Codes: C07K 14/705 Receptors; Cell surface antigens; Cell surface determinants tumour specific antigens C07K14/4748
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: H04L 43/0888 Throughput
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Systems, methods and apparatus for analyzing Internet traffic. In an aspect, a method receives at a server from a client device a report request for a report related to web site traffic; in response to the report request, sends from the server web site traffic data and application code to the client device. The application code has instructions that cause the client device to: generate a report to display the web site traffic data, time the display of the web site traffic data, periodically request updated web site traffic data according to the time of the display, and update the report with the updated web site traffic data; and the method sends from the server to the client device the updated web site traffic data in response to the request for updated web site traffic data.

Claims

1. A computer-implemented method, comprising:

accessing visitor data from a log file that is separate from a persistent database storing visitor data content describing web traffic activity for a web server, the visitor data comprising log lines, each log line including data specifying a particular web page requested by a visitor, a time stamp associated with the request, and a visitor identifier;

for each log line;

determining, from the time stamp, whether the time stamp corresponds to a current period defining a first time period during which visitors visited the website and for which visitor data is stored in a buffer or a new period defining a second time period during which visitors visited the website and for which visitor data is not stored in the buffer, wherein the first time period is different from the second time period, and only in response to the time stamp corresponding to the new period, loading, from the persistent database, the buffer with all visitor data content for the new period;

accessing a visitor record that corresponds to the visitor identifier, the visitor record stored in the buffer; and

updating the visitor record corresponding to the visitor identifier with the data specifying the particular web page requested by a visitor, the time stamp associated with the request;

wherein the data specifying the particular web page requested by a visitor is inserted into the visitor record at an ordinal position based on the time stamp for the particular web page requested relative to time stamps of other web pages requested specified by other log lines for the visitor identifier.

2. The computer-implemented method of claim 1, wherein the visitor record defines visitor footprint data related to the visitor, the visitor footprint data including one or more of browser data, referral data, domain data and visitor path data.

3. The computer-implemented method of claim 1, further comprising:

for each log line, determining the visitor identifier; and

for each visitor identifier, generating a corresponding hash value in hash table for the visitor identifier; and

wherein accessing a visitor record that corresponds to the visitor identifier comprises accessing the hash table with a hash value corresponding to the visitor identifier.

4. The computer-implemented method of claim 3, further comprising:

periodically determining whether the hash table meets a predetermined fullness threshold, the fullness threshold defining a partially empty measure for the hash table;

in response to determining the hash table meets the predetermined fullness threshold, increasing the size of the hash table.

5. The computer-implemented method of claim 1, further comprising, in response to determining a completion of processing of log lines in the visitor data, persisting data stored in the buffer to a persistent database.

6. A system, comprising:

one or more server computers; and

a memory subsystem storing instructions executable by one or more server computers that upon such execution cause the one or more server computers to perform operations comprising;

accessing visitor data from a log file that is separate from a persistent database storing visitor data content describing web traffic activity for a web server, the visitor data comprising log lines, each log line including data specifying a particular web page requested by a visitor, a time stamp associated with the request, and a visitor identifier;

for each log line;

determining, from the time stamp, whether the time stamp corresponds to a current period defining a first time period during which visitors visited the website and for which visitor data is stored in a buffer or a new period defining a second time period during which visitors visited the website and for which visitor data is not stored in the buffer, wherein the first time period is different from the second time period, and only in response to the time stamp corresponding to the new period, loading, from the persistent database, the buffer with all visitor data content for the new period;

accessing a visitor record that corresponds to the visitor identifier, the visitor record stored in the buffer; and

updating the visitor record corresponding to the visitor identifier with the data specifying the particular web page requested by a visitor, the time stamp associated with the request;

wherein the data specifying the particular web page requested by a visitor is inserted into the visitor record at an ordinal position based on the time stamp for the particular web page requested relative to time stamps of other web pages requested specified by other log lines for the visitor identifier.

7. The system of claim 6, wherein the visitor record defines visitor footprint data related to the visitor, the visitor footprint data including one or more of browser data, referral data, domain data and visitor path data.

8. The system of claim 6, the operations further comprising:

for each log line, determining the visitor identifier; and

for each visitor identifier, generating a corresponding hash value in hash table for the visitor identifier; and

wherein accessing a visitor record that corresponds to the visitor identifier comprises accessing the hash table with a hash value corresponding to the visitor identifier.

9. The system of claim 8, the operations further comprising:

periodically determining whether the hash table meets a predetermined fullness threshold, the fullness threshold defining a partially empty measure for the hash table;

in response to determining the hash table meets the predetermined fullness threshold, increasing the size of the hash table.

10. The system of claim 6, the operations further comprising, in response to determining a completion of processing of log lines in the visitor data, persisting data stored in the buffer to a persistent database.

11. A non-transitory computer readable storage device and including instructions that cause a computer to perform operations comprising:

accessing visitor data from a log file that is separate from a persistent database storing visitor data content describing web traffic activity for a web server, the visitor data comprising log lines, each log line including data specifying a particular web page requested by a visitor, a time stamp associated with the request, and a visitor identifier;

for each log line;

determining, from the time stamp, whether the time stamp corresponds to a current period defining a first time period during which visitors visited the website and for which visitor data is stored in a buffer or a new period defining a second time period during which visitors visited the website and for which visitor data is not stored in the buffer, wherein the first time period is different from the second time period, and only in response to the time stamp corresponding to the new period, loading, from the persistent database, the buffer with all visitor data content for the new period;

accessing a visitor record that corresponds to the visitor identifier, the visitor record stored in the buffer; and

updating the visitor record corresponding to the visitor identifier with the data specifying the particular web page requested by a visitor, the time stamp associated with the request;

wherein the data specifying the particular web page requested by a visitor is inserted into the visitor record at an ordinal position based on the time stamp for the particular web page requested relative to time stamps of other web pages requested specified by other log lines for the visitor identifier.

12. The non-transitory computer readable storage device of claim 11, wherein the visitor record defines visitor footprint data related to the visitor, the visitor footprint data including one or more of browser data, referral data, domain data and visitor path data.

13. The non-transitory computer readable storage device of claim 11, the operations further comprising:

for each log line, determining the visitor identifier; and

for each visitor identifier, generating a corresponding hash value in hash table for the visitor identifier; and

wherein accessing a visitor record that corresponds to the visitor identifier comprises accessing the hash table with a hash value corresponding to the visitor identifier.

14. The non-transitory computer readable storage device of claim 13, the operations further comprising:

periodically determining whether the hash table meets a predetermined fullness threshold, the fullness threshold defining a partially empty measure for the hash table;

in response to determining the hash table meets the predetermined fullness threshold, increasing the size of the hash table.

15. The non-transitory computer readable storage device of claim 11, the operations further comprising, in response to determining a completion of processing of log lines in the visitor data, persisting data stored in the buffer to a persistent database.