Systems and methods for clickstream analysis to modify an off-line business process involving determining related or complementary items

• US 7,797,371 B2
• Filed: 10/24/2007
• Issued: 09/14/2010
• Est. Priority Date: 03/07/2002
• Status: Active Grant

• Current Assignee: comScore, Inc.
• Original Assignee: Compete, Inc. (comScore, Inc.)
• Inventors: Cancel, David, Singh, Man Jit
• Primary Examiner(s): Najjar; Saleh
• Assistant Examiner(s): Henry; Maria Georges
• Application Number:
  US11/923,610
• Publication Number:
  US 20080183867A1
• Time in Patent Office: 1,056 Days
• Field of Search: 709/201, 705/10, 705/26
• US Class Current: 709/201
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 15/17 using an input/output type connection, e.g. channel, I/O port
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: G06Q 20/20 Point-of-sale [POS] network systems
• CPC Class Codes: G06Q 20/201 Price look-up processing, e.g. updating
• CPC Class Codes: G06Q 20/203 Inventory monitoring
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0204 Market segmentation
• CPC Class Codes: G06Q 30/0224 based on user history
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0243 Comparative campaigns
• CPC Class Codes: G06Q 30/0244 Optimization
• CPC Class Codes: G06Q 30/0246 Traffic
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04H 60/46 for recognising users' preferences
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/146 : Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Methods and systems are provided herein for the analysis of clickstream data of online users. The analysis methods and systems allow for the creation of new offline business methods based on online consumer behavior. In embodiments, the output file enables a third party to determine related products for product placement and arrangement in a retail facility to enhance revenue from purchases of related products.

Claims

1. A method of processing data at a host computer, comprising:

receiving a data file containing records of a plurality of HTTP (HyperText Transfer Protocol) transactions of a plurality of users, each of the HTTP transactions including at least one URL (Uniform Resource Locator);

converting the data file into a common file format;

cleansing the data file in the common file format by applying a plurality of URL rules to remove session identifiers while accounting for the use of proxies, thereby producing a cleansed data file containing at least one modified URL;

performing a panel selection process, comprising;

conducting a periodic survey of a subset of Internet users to determine characteristics of an Internet community that includes the subset of Internet users, the characteristics including demographic characteristics;

selecting a panel of users from the subset of Internet users, the panel of users having a targeted combination of the demographic characteristics;

retaining the data file only when it contains a user identifier of a user in the panel of users;

using a computer, performing a transformation process, comprising;

removing from the data file those URLs that do not contain the user identifier;

determining which URLs in the data file belong to a session of the user in the panel of users;

assigning a session identifier to the session;

creating a plurality of session data files, each of which contain the user identifier, a time stamp, the URLs in the data file that belong to the session, and the session identifier;

decomposing the URLs in each of the plurality of session data files so that the URLs in each of the plurality of session data files are decomposed URLs; and

hashing the decomposed URLs so that the decomposed URLs are hashed, decomposed URLs;

calculating a metric of user behavior based upon the hashed, decomposed URLs;

merging the metric of user behavior into a file containing a plurality of metrics of user behavior; and

transmitting to a remote machine the file containing the plurality of metrics of user behavior, wherein the file containing the plurality of metrics of user behavior enables a third party to determine related products for product placement and arrangement in a retail facility to enhance revenue from purchases of the related products.

2. The method of claim 1, wherein the related products are media content.

3. The method of claim 1, wherein the file containing a plurality of metrics of user behavior further enables the third party to determine a complementary product or service.

4. The method of claim 1, wherein the file containing a plurality of metrics of user behavior further enables the third party to determine a retail location based on aspects of competitive sales in a location geography.

5. The method of claim 4, wherein one of the aspects of competitive sales in a location geography is a competitor'"'"'s online shopper density.

6. The method of claim 1, wherein the file containing a plurality of metrics of user behavior further enables the third party to select a retail location based on a retail location of a complementary product or service.

7. The method of claim 1, wherein at least one of the URL rules is generated by the host computer.

8. The method of claim 1, wherein the host computer comprises software modules and a cluster of machines that run parallel calculations on each of the plurality of session data files.

9. The method of claim 8, wherein the parallel calculations include the calculating of the metric of user behavior based upon the hashed, decomposed URLs.

10. A host computer, comprising:

a data collection facility that receives a data file containing records of a plurality of HTTP (HyperText Transfer Protocol) transactions of a plurality of users, each of the HTTP transactions including at least one URL (Uniform Resource Locator);

checks for integrity of the data file and the records; and

converts the data file into a common file format;

a pre-calculation facility comprising a computer, that performs a cleansing process, a panel selection process, and a transformation process, wherein

the cleansing process comprises;

cleansing the data file in the common file format by applying a plurality of URL rules to remove session identifiers while accounting for the use of proxies, thereby producing a cleansed data file containing at least one modified URL;

the panel selection process comprises;

conducting a periodic survey of a subset of Internet users to determine characteristics of an Internet community that includes the subset of Internet users, the characteristics including demographic characteristics;

selecting a panel of users from the subset of Internet users, the panel of users having a targeted combination of the demographic characteristics;

retaining the data file only when it contains a user identifier of a user in the panel of users;

the transformation process comprises;

removing from the data file those URLs that do not contain the user identifier;

determining which URLs in the data file belong to a session of the user in the panel of users;

assigning a session identifier to the session;

creating a plurality of session data files, each of which contain the user identifier, a time stamp, the URLs in the data file that belong to the session, and the session identifier;

decomposing the URLs in each of the plurality of session data files so that the URLs in each of the plurality of session data files are decomposed URLs; and

hashing the decomposed URLs so that the decomposed URLs are hashed, decomposed URLs; and

a calculation facility that calculates a metric of user behavior based upon the hashed, decomposed URLs, performs a post-calculation process, and transmits to a remote machine a file containing a plurality of metrics of user behavior, wherein the file containing a plurality of metrics of user behavior enables the third party to determine related products for product placement and arrangement in a retail facility to enhance revenue from purchases of related products, and wherein

the post-calculation process comprises;

merging the metric of user behavior into a file containing a plurality of metrics of user behavior.

11. The host computer of claim 10, wherein the related products are media content.

12. The host computer of claim 10, wherein the file containing a plurality of metrics of user behavior further enables the third party to determine a complementary product or service.

13. The host computer of claim 10, wherein the file containing a plurality of metrics of user behavior further enables the third party to determine a retail location based on aspects of competitive sales in a location geography.

14. The host computer of claim 13, wherein one of the aspects of competitive sales in a location geography is a competitor'"'"'s online shopper density.

15. The host computer of claim 10, wherein the file containing a plurality of metrics of user behavior further enables the third party to select a retail location based on a retail location of a complementary product or service.

16. The host computer system of claim 10, wherein at least one of the URL rules is generated by the host computer system.

17. The host computer system of claim 10, wherein the host computer system comprises software modules and a cluster of machines that run parallel calculations on each of the plurality of session data files.

18. The host computer system of claim 17, wherein the parallel calculations include the calculating of the metric of user behavior based upon the hashed, decomposed URLs.