METHOD AND SYSTEM FOR CHARACTERIZING WEB CONTENT

• US 20110029505A1
• Filed: 07/31/2009
• Published: 02/03/2011
• Est. Priority Date: 07/31/2009
• Status: Abandoned Application

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Micro Focus LLC (Open Text Corporation)
• Inventors: Scholz, Martin B., Rajaram, Shyam Sundar, Lukose, Rajan
• Application Number:
  US12/533,717
• Publication Number:
  US 20110029505A1
• Time in Patent Office: Days
• US Class Current: 707/711
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]

Abstract

An exemplary embodiment of the present invention provides a method of processing Web activity data. The method includes obtaining a database of clickstream data comprising a user identifier corresponding with a user ID and a uniform resource locator (URL) corresponding with a Web page visited from the user ID. The method also includes generating a plurality of features based on the URL. Further, the method includes generating a data structure comprising the user ID and the feature. The method also includes generating segment information from the data structure based on the similarity of a URL visitation pattern across different user IDs, wherein each segment in the segment information comprises one or more user IDs and one or more features.

Claims

1. A method of processing Web activity data, comprising:

retrieving a database of clickstream data comprising a user identifier (user ID) and a uniform resource locator (URL) corresponding to a Web page;

truncating the URL to identify a feature of the URL;

building a data structure comprising the user ID and the feature; and

generating segment information from the data structure based on a similarity of a URL visitation pattern across different user IDs, wherein each segment in the segment information comprises one or more of the different user IDs and one or more features.

2. The method of claim 1, wherein truncating the URL to identify a feature generates lower-level URLs with gradually increasing levels of abstraction compared to the URL.

3. The method of claim 1, wherein truncating the URL to identify a feature comprises truncating the URL at a delimiter including at least one of a slash, ampersand, an at sign, a question mark, a colon, a number sign, or an equals sign.

4. The method of claim 1, wherein truncating the URL to identify a feature comprises extracting keywords from the URL of a search engine.

5. The method of claim 1, comprising eliminating the feature based on a count of the different user IDs that have visited the Web page corresponding to the feature.

6. The method of claim 5, wherein eliminating the feature comprises specifying a count N and eliminating the feature if the Web page corresponding to the feature has been visited by less than N of the different user IDs.

7. The method of claim 1, wherein generating the segment information comprises processing the data structure using at least one of clustering, co-clustering, or information-theoretic co-clustering.

8. The method of claim 1, comprising loading the segment information to a database that is accessible to a Website, wherein the Website uses the segment information to determine the content of a Web page.

9. The method of claim 8, wherein the segment information is used by the Website to provide an advertisement to a user ID that is accessing the Website.

10. The method of claim 1, comprising assigning a category name to each segment in the segment information based on an apparent subject matter encompassed by the segment.

11. A computer system, comprising:

a processor that is adapted to execute machine-readable instructions;

a storage device that is adapted to store data, the data comprising a database of clickstream data; and

a memory device that stores instructions that are executable by the processor, the instructions comprising;

a feature generator adapted to receive a URL from the database of clickstream data and generate one or more features based on the URL;

a data structure builder adapted to analyze the clickstream data to identify a user ID and one or more features that correspond with the user ID and to enter the user ID and the one or more features into a data structure; and

a segment information generator adapted to process the data structure to generate segments that group user IDs and the one or more features based on a similarity of a visitation pattern.

12. The computer system of claim 11, wherein the feature generator truncates the URL at each forward slash in the URL to provide the one or more features.

13. The computer system of claim 11, wherein the feature generator truncates the URL at each dot in a domain name of the URL to provide the one or more features.

14. The computer system of claim 11, wherein the instructions comprise a feature eliminator that is configured to remove features from the data structure that have a level of support that is too high or too low.

15. The computer system of claim 14, wherein the feature eliminator is adapted to remove features from the data structure that are supported by less than a minimum number of visitors.

16. The computer system of claim 11, wherein the segment information generator is adapted to generate the groupings via co-clustering.

17. The computer system of claim 11, wherein each of the segments comprises a list of Web page URLs and a corresponding list of user IDs that have accessed the Web page addresses.

18. A tangible, computer-readable medium, comprising:

code adapted to receive a URL from a database of clickstream data and generate one or more features based on the URL;

code adapted to receive a user ID from the clickstream data and a plurality of features from the feature generator that correspond with the user ID and enter the user ID and features into a data structure; and

code adapted to process the data structure to generate groupings of user IDs and features based on a similarity of a visitation pattern.

19. The tangible, computer-readable medium of claim 18, comprising code adapted to truncate a URL to produce a plurality of features comprising new URLs with increasing levels of abstraction.

20. The tangible, computer-readable medium of claim 18, comprising code adapted eliminate the new URLs from the data structure if the new URLs are not matched with a preselected number of user IDs.