system and method for detecting and reporting online activity using real-time content-based network monitoring

• US 20020128925A1
• Filed: 12/11/2001
• Published: 09/12/2002
• Est. Priority Date: 12/11/2000
• Status: Active Grant

• Current Assignee: Network Monitoring LLC
• Original Assignee: Peter K. Trzyna
• Inventors: Angeles, Patrick
• Granted Patent: US 9,058,416 B2
• Time in Patent Office: Days
• US Class Current: 705/26
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: G06F 16/986 Document structures and storage, e.g. HTML extensions
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 40/12 Accounting
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 61/10 of different types
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 63/0281 Proxies
• CPC Class Codes: H04L 63/029 Firewall traversal, e.g. tunnelling or, creating pinholes
• CPC Class Codes: H04L 63/0823 : using certificates cryptographic mechanisms or cryptographic arrangements for entity authentication involving certificates H04L9/3263
• CPC Class Codes: H04L 63/166 : at the transport layer
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 69/329 : in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 : Network security protocols

Abstract

A computerized system and method for tracking and reporting online activity across a plurality of clients and servers is disclosed that intercepts and logs secure and non-secure HTTP request and response pages, analyzes each of the received page records, associates each page record to an event type based on user-defined parameters, identifies and extracts user-defined attributes of each page record based on its event type, and reports on the occurrence of the event along with its associated attributes.

Claims

1. A microprocessor-controlled system for tracking electronic commerce activity across a distributed computer network, the system comprising:

random access memory comprising;

software executable on a microprocessor for data collection and acquisition; and

software executable on a microprocessor for data management and processing; and

a microprocessor for execution of the data collection and acquisition software and the data management software.

2. The system of claim 1, wherein the distributed computer network comprises a plurality of clients and servers.

3. A method for continuous event monitoring over a distributed computing network, comprising:

(a) rewriting an original URL such that a browser request from a browser to the original URL is received by a proxy;

(b) receiving the browser request at the proxy;

(c) sending a proxy request to the original URL;

(d) receiving a response from a host of the original URL;

(e) rewriting all URLs embedded in the response such that a browser request to each of the embedded URLs is received by the proxy; and

(f) serving the response to the browser.

4. A method for tracking user activity in a distributed computer network, comprising the steps of:

(a) capturing data relating to user activity on the distributed network;

(b) storing captured data in a central database;

(c) extracting relevant data from the database based upon pre-determined selection parameters to form parameter data sets;

(d) dynamically maintaining the pre-determined selection parameters based upon revisable, operator-defined instructions on how to select and extract information from a text page; and

(e) analyzing the parameter data sets to predict future user activity.

5. The method of claim 4, wherein the distributed network comprises a plurality of clients and servers.

6. The method of claim 4, wherein the text page is encoded with HTML and accessible over the Internet.

7. The method of claim 4, wherein the user activity occurs over a secure communications channel.

8. The method of claim 7, wherein the secure communications channel is Secure Sockets Layer (SSL).

9. The method of claim 4, wherein the data relating to user activity is captured by using an URL-rewriting proxy agent.

10. The method of claim 4, wherein the data relating to user activity is captured by using an HTTP-proxy agent.

11. The method of claim 4, wherein the data relating to user activity comprises purchase amounts, purchased item identification, travel dates, e-mail addresses, or zip codes.

12. The method of claim 4, wherein the method further comprises the step of revising the instructions on how to select and extract information from the text page to create selection parameters based on analysis of parameter data sets.

13. The method of claim 4, wherein the method includes the further step of generating a report on the analysis of parameter data sets.

14. The method of claim 4, wherein the method further includes repeating steps (a) through (e).

15. The method of claim 4, wherein the user activity comprises an activity selected from the group consisting of purchases of goods and/or services, online shopping cart abandonment, credit card type usage, user account creation, and contest participation.

17. The system as recited in claim 16, wherein the original URL comprises a request protocol, a hostname, a port number and a path.

20. The system of claim 19, wherein the means for capturing data relating to user activity is a server.

21. The system of claim 19, wherein the distributed computer network comprises a plurality of clients and servers.

22. The system of claim 19, wherein the system further comprises a means for generating reports comprising results of the analysis of the relevant data.

24. The method of claim 23, wherein, in the event that thresholds are not met, the method comprises the further steps of:

(I) determining if user-defined parameters require amendment;

(j) amending the user-defined parameters; and

(k) validating the amended parameters against recorded HTTP transmissions.

26. The method for assisting in tracking and reporting online activity of claim 25, the method comprising the further step of recording an HTTP transmission stream for creating, amending or validating user-defined parameters for use in the identification of HTTP transmission records.

27. The method for assisting in tracking and reporting online activity of claim 25, the method comprising the further step of extracting attribute data from identified HTTP transmission records.

28. The method for assisting in tracking and reporting online activity of claim 25, wherein one or more mirror systems are established for the creation, amendment and validation of user-defined parameters for the identification, attribute data extraction, and analysis of HTTP transmissions.

29. The method for assisting in tracking and reporting online activity of claim 27, further comprising a step of synchronizing user-defined parameters between the mirror systems.

30. The method for assisting in tracking and reporting online activity of claim 27, further comprising a step of transferring HTTP transmission records to the one or more mirror systems for the purpose of creating, amending or validating user-defined parameters for use in the identification of HTTP transmission records or extraction of attribute data from identified HTTP transmission records.

16. A system for continuous event monitoring over a distributed computer network, comprising a server for detection, identification and analysis of a plurality of events, the server having:

(a) a memory for storing records, and

(b) a processor operatively connected to the memory, the memory storing instructions for rewriting an original URL as a proxy URL in order to cause a browser to send a request to the server, wherein when the browser sends the request to the server, the server sends a server request to the original URL, receives a response and serves the response to the browser while collecting data.

18. A computer-readable medium whose contents cause a distributed computer network to track user activity, the distributed computer network having client computers connected to servers, by performing the steps of:

(a) capturing data relating to user activity in a central database;

(b) extracting relevant data from the database based upon profiles;

(c) maintaining the profiles based upon instructions on how to identify information from a Web page; and

(d) analyzing the relevant data to predict future user activity.

19. A system for tracking user activity in a distributed computer network, comprising:

(a) means for capturing data relating to user activity in a central database;

(b) means for extracting relevant data from the database based upon profiles;

means for maintaining the profiles based upon instructions on how to identify information from a text page; and

(c) means for analyzing the relevant data to predict future user activity.

23. A method for tracking and reporting user activity across a distributed network comprising a plurality of clients and one or more servers, the method comprising the steps of:

(a) capturing at least one HTTP transmission record between at least one browser on a client and at least one Web site on a server;

(b) analyzing each of said recorded HTTP transmissions to determine an identity for the transmission based on user-defined parameters;

(c) extracting attribute data from each of said identified Web pages based on user-defined parameters;

(d) detecting an occurrence of an event;

(e) composing attributes of the event from attributes extracted from at least one HTTP transmission record;

(f) generating a report relating to data resulting from the capture, identification, attribute data extraction and analysis of the HTTP transmission records;

(g) comparing statistical information contained in the report to predefined thresholds; and

(h) responding when said thresholds are not met.

25. A method for assisting in tracking and reporting online activity across a distributed network comprising a plurality of clients and one or more servers, the method comprising the steps of:

(a) creating, amending and validating user-defined parameters to assist in the identification of HTTP transmission records;

(b) creating, amending and validating user-defined parameters to assist in the extraction of attribute data from identified HTTP transmission records;

(c) determining if user-defined parameters to assist in the identification of HTTP transmission records require amendment;

(d) determining if user-defined parameters to assist in the extraction of attribute data from identified HTTP transmission records require amendment; and

(e) generating reports when amendments to user-defined parameters are made.