Parsing navigation information to identify interactions based on the times of their occurrences

• US 7,107,338 B1
• Filed: 12/05/2001
• Issued: 09/12/2006
• Est. Priority Date: 12/05/2001
• Status: Active Grant

• Current Assignee: Revenue Science
• Original Assignee: Revenue Science
• Inventors: Nareddy, Krishnamohan, Uppala, Radha Krishna
• Primary Examiner(s): Najjar, Saleh
• Assistant Examiner(s): KOROBOV, VITALI A
• Application Number:
  US10/005,183
• Time in Patent Office: 1,742 Days
• Field of Search: 709/223, 709/224, 709/203, 709/200, 709/202, 709/229, 709217-219, 709/245, 715/853, 715/854, 707/1, 707/3, 707/6, 707/100
• US Class Current: 709/224
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A method, system and computer-readable medium for analyzing interaction or usage data, such as for customers, is described. Various data parsing information may be defined and used as part of the analysis, such as by using customer-specific information to identify various occurrences of interest. For example, the parser component can use data defining customer-specific categories of content set items and customer-specific types of events of interest. Such high-level types of occurrences can be specified in a variety of ways, such as by using a combination of a logical web site, one or more URIs corresponding to web pages, and/or one or more query strings. In addition, in order to associate the appropriate data parsing information with data to be processed, the data parsing information can also include version information that specifies when it is applicable. The data parsing information may also map actual web sites to logical sites.

Claims

1. A computer-implemented method for using predefined parsing information to analyze web site navigation data in order to identify occurrences of interest, the web site navigation data including information indicating times of occurrence and the predefined parsing information including information indicating occurrence times for which the predefined parsing information is effective, the method comprising:

for each of multiple distinct web sites each having multiple web pages,

receiving web site navigation data associated with the web site that has multiple entries each containing information related to a request for a web page of the web site and to a response to that request from a web site server for the web site, the contained information including a time of occurrence;

retrieving predefined parsing information associated with the web site that includes multiple distinct definitions of logical sites, multiple distinct definitions of event types, and multiple distinct definitions of category types, each logical site definition specifying an IP address and port number used by a web site server to provide at least some of the web pages of the web site and specifying times of occurrence to which the logical site definition applies, each event type definition specifying a type of request for a web page of the web site and specifying times of occurrence to which the event type definition applies, each category type definition specifying a group of web pages of the web site that are each related to a category and specifying times of occurrence to which the category type definition applies, and each of the event type definitions and category type definitions further specifying one of the defined logical sites;

for each entry of the received web site navigation data,

analyzing the information contained in the entry to determine if the web site server that provided the related response matches any of the logical site definitions by having used the IP address and the port number specified by that logical site definition and by having a time of occurrence included in the contained information that is within the specified times of occurrence for that logical site definition; and

if a logical site is determined to match the web site server, further analyzing the information contained in the entry by

for each of the event type definitions that specifies the matching logical site, when the information contained in the entry indicates that the related request is of the type specified by the event type definition and the time of occurrence included in the contained information is within the specified times of occurrence for the event type definition, storing an indication of an occurrence for the web site of a request of the type specified by that event type; and

for at least one of the category type definitions that specifies the matching logical site, when the information contained in the entry indicates that the related request is for a web page in the group specified by that category type definition and the time of occurrence included in the contained information is within the specified times of occurrence for that category type definition, storing an indication of an occurrence for the web site of a request for a web page that is a member of the group specified by that category type;

receiving a request from an operator of the web site to provide information for the web site about occurrences of specified event types or category types;

retrieving in response the stored indications of occurrences related to the specified event types or category types for the web site; and

providing the retrieved information to the operator,

so that the operators of the multiple web sites can receive information about occurrences of interest for their web sites.

2. The method of claim 1 wherein the time of occurrence included in the contained information for each entry corresponds to a date of the related request for the entry.

3. The method of claim 1 wherein the time of occurrence included in the contained information for each entry corresponds to a date of the related response for the entry.

4. The method of claim 1 wherein the times of occurrence for the logical site definitions, event type definitions and category type definitions are each specified to include at least one range of dates.

5. The method of claim 1 wherein the times of occurrence for the logical site definitions, event type definitions and category type definitions are each specified using an indication of one of multiple versions that each have associated effective times.

6. The method of claim 1 wherein for at least some of the event types and category types, the parsing information includes multiple definitions for that event type or category type whose specified times of occurrence differ.

7. The method of claim 6 wherein the received web site navigation data for at least one of the web sites includes multiple entries each containing information that indicates that the related request for that entry is of one of the event types having multiple definitions with differing specified times of occurrence or each containing information that indicates that the related request is for a web page in the group specified by one of the category types having multiple definitions with differing specified times of occurrence, those multiple entries having varying included times of occurrences, wherein the further analyzing of the information contained in those multiple entries includes storing indications of occurrences of that one event type or that one category type for each of those multiple entries, and wherein each of the multiple definitions for that one event type or one category type are used when determining to store the indications of the occurrences for at least one of those multiple entries.

8. The method of claim 1 including, after the retrieving of the predefined parsing information associated with one of the web sites and before the analyzing of the information contained in the entries of the received web site navigation data associated with that one web site:

separating the logical site definitions, event type definitions and category type definitions into groups of definitions having the same specified times of occurrences; and

for each entry of the received web site navigation data associated with that one web site, selecting one of the groups of definitions for the entry based on the time of occurrence included in the contained information for the entry being within the specified times of occurrence for that one group of definitions, the selecting of the one group of definitions such that the analyzing and further analyzing of the information contained in that entry will only use definitions in the one group of definitions.

9. The method of claim 1 wherein the predefined parsing information associated with one of the web sites further includes at least one exclusion definition that specifies a type of request or response and specifies times of occurrence of requests or responses of that type to which the exclusion definition applies, and wherein the analyzing of the information contained in an entry of the web site navigation data associated with that web site is not performed if the related request or response for that entry is of a type that matches a type specified for one of the exclusion definitions and if the time of occurrence included in the contained information for that entry is within the times of occurrences specified for that one exclusion definition.

10. The method of claim 1 wherein the contained information for the entries of the received web site navigation data for one of the web sites further include an indication of a user from whom the related request was received, wherein the predefined parsing information associated with one of the web sites further includes multiple user definitions that each specify information about a user from whom a request may be received and specify times of occurrence to which the user definition applies, and including, for each entry of the received web site navigation data for that one web site:

analyzing the information contained in the entry to determine if the related request was from a user that matches one of the user definitions that specifies times of occurrence that include the time of occurrence included in the contained information; and

when the information contained in the entry indicates that the related request was from a user that matches one of the user definitions that specifies times of occurrence that include the time of occurrence included in the contained information, associating an indication of that user definition with any stored indications for that entry of occurrences of requests of types specified by event types and of requests for web pages that are members of groups specified by category types.

11. The method of claim 1 wherein the contained information for the entries of the received web site navigation data for one of the web sites further include an indication of a product to which the related request corresponds, wherein the predefined parsing information associated with one of the web sites further includes multiple product definitions that each specify information about a product to which received requests may correspond and specify times of occurrence to which the product definition applies, and including, for each entry of the received web site navigation data for that one web site:

analyzing the information contained in the entry to determine if the related request corresponds to a product that matches one of the product definitions that specifies times of occurrence that include the time of occurrence included in the contained information; and

when the information contained in the entry indicates that the related request was corresponds to a product that matches one of the product definitions that specifies times of occurrence that include the time of occurrence included in the contained information, associating an indication of that product definition with any stored indications for that entry of occurrences of requests of types specified by event types and of requests for web pages that are members of groups specified by category types.

12. The method of claim 1 including, before the receiving of the web site navigation data for a web site, generating the parsing information associated with the web site based at least in part on a current version of the web site servers that can provide web pages of the web site, of the types of requests for web pages to which those web site server can respond, and of each group of the web pages that are each related to a category.

13. The method of claim 12 including, after the generating of the parsing information associated with the web site:

receiving an indication of changes to the web site servers that can provide web pages of the web site, to the types of requests for web pages to which those web site server can respond, and/or to the groups of the web pages that are each related to a category; and

generating an updated version of the parsing information associated with the web site based at least in part on the indicated changes.

14. The method of claim 1 wherein the receiving of the web site navigation data associated with a web site includes retrieving at least one log file from at least one web site server for the web site, the retrieved log files containing the web site navigation data.

15. The method of claim 1 wherein the received request from an operator of one of the web sites further specifies effective times such that the information to be provided is for occurrences of the specified event types or category types that took place during the effective times, and wherein the stored indications that are retrieved in response are for those occurrences.

16. The method of claim 1 wherein the operator of one of the web sites from whom a request is received is at a remote location, and wherein the providing of the retrieved information to the operator includes generating a report that includes the retrieved information and sending the generated report to the remote location for presentation to the operator.

17. The method of claim 1 wherein the operators of the multiple web sites are customers, and wherein the analyzing of the web site navigation data entries for the web sites is performed as a service for the customers.

18. A computer-implemented method for analyzing interaction data for a web site to identify occurrences of defined types of interactions using information about times of the occurrences, comprising:

receiving an indication of multiple interaction data entries each containing information about an interaction with a web site, the contained information for each entry including a specified URL and a time of occurrence related to the interaction, each of the specified URLs optionally including a URL path portion and optionally including a query string portion;

receiving an indication of multiple interaction type definitions that each specify a type of interaction and each have associated interaction times during which the interaction type definition is applicable to interactions that occur, have a URL path pattern capable of matching at least one URL path related to the interaction type, and have a query string pattern capable of matching at least one query string related to the interaction type; and

for each entry,

analyzing the entry to determine whether the entry matches one of the interaction type definitions by containing information about an interaction of the type specified by that one interaction type definition and by including a time of occurrence that is among the interaction times during which that one interaction type definition is applicable, the matching such that the specified URL in the contained information for the entry includes a URL path portion that matches the URL path pattern specified in that one interaction type definition and includes a query string portion that matches the query string pattern specified in that one interaction type definition; and

when it is determined that the entry matches one of the interaction type definitions, storing an indication of an occurrence of that interaction type for the web site.

19. The method of claim 18 wherein the contained information about each interaction further includes information related to a manner of identifying a web site server with which the interaction occurred, wherein each of the interaction type definitions is associated with a logical site definition that specifies a manner of identifying a web site server related to the web site and that specifies times when that logical site definition is applicable, and wherein the determining that an entry matches an interaction type definition further includes determining that the entry includes information related to the manner of identifying the web site server that matches the manner of identifying a web site server specified by the logical site definition associated with that interaction type definition and that the time of occurrence included in the contained information for the entry is among the interaction times during which that logical site definition is specified to be applicable.

20. The method of claim 18 wherein at least some of the interaction type definitions include multiple interaction patterns that each specify a distinct combination of a URL path pattern capable of matching at least one URL path and a query string pattern capable of matching at least one query string, and wherein an entry is determined to match an interaction type definition having multiple interaction patterns if, for any of the interaction patterns, the specified URL in the contained information for the entry includes a URL path portion that matches the URL path pattern specified in that interaction pattern and includes a query string portion that matches the query string pattern specified in that interaction pattern.

21. The method of claim 18 wherein the associated interaction times for each of the interaction type definitions includes a range of dates.

22. The method of claim 18 wherein each of the interaction type definitions specifies a version identifier that is associated with a range of time, and wherein the associated interaction times for each of the interaction type definitions is the range of time associated with the version identifier specified by that interaction type definition.

23. The method of claim 18 wherein each of the interaction type definitions is an event type definition.

24. The method of claim 18 wherein each of the interaction type definitions is a category type definition.

25. The method of claim 18 wherein at least some of the specified URLs include an empty query string portion or an empty URL path portion.

26. The method of claim 18 including presenting information for at least some of the stored indications.

27. A computer-readable medium containing instructions that when executed cause a computer system to analyze data for a web site to identify occurrences of defined types of interactions using information about times of the occurrences, by performing a method comprising:

receiving an indication of multiple data entries each containing information about an interaction with a web site, the contained information for each entry including a specified URL and a time of occurrence related to the interaction, each of the specified URLs optionally including a URL path portion and optionally including a query string portion;

receiving an indication of multiple interaction type definitions that each specify a type of interaction and each have associated interaction times during which the interaction type definition is applicable to interactions that occur, have a URL path pattern capable of matching at least one URL path related to the interaction type, and have a query string pattern capable of matching at least one query string related to the interaction type; and

for each entry,

analyzing the entry to determine whether the entry matches one of the interaction type definitions by containing information about an interaction of the type specified by that one interaction type definition and by including a time of occurrence that is among the interaction times during which that one interaction type definition is applicable, the matching such that the specified URL in the contained information for the entry includes a URL path portion that matches the URL path pattern specified in that one interaction type definition and includes a query string portion that matches the query string pattern specified in that one interaction type definition; and

when it is determined that the entry matches one of the interaction type definitions, storing an indication of an occurrence of that interaction type for the web site.

28. A method for analyzing interaction data for a web site to identify occurrences of defined types of interactions using information about times of the occurrences, the method comprising:

receiving an indication of multiple interaction data entries each containing information including a URL corresponding to a web site that was specified during a request and including a time of occurrence related to the request, each of the specified URLs including a URL path portion and a query string portion;

receiving an indication of multiple interaction type definitions that each specify a type of interaction, each interaction type definition including multiple distinct interaction patterns that each have associated interaction occurrence times for which the interaction pattern is effective and that each specify a distinct combination of a URL path pattern capable of matching at least one URL path and a query string pattern capable of matching at least one query string; and

for each entry,

analyzing the entry to determine whether the entry matches one of the interaction type definitions by containing information about an interaction of the type specified by that one interaction type definition, the matching such that, for at least one of the interaction patterns included in that one interaction type definition, the information contained in the entry includes a specified URL whose URL path portion and query string portion match the URL path pattern and the query string pattern specified in that interaction pattern and includes a time of occurrence that is among the interaction occurrence times for which that interaction pattern is effective; and

when it is determined that the entry matches one of the interaction type definitions, storing an indication of an occurrence of that interaction type for the web site.

29. The method of claim 28 wherein the contained information for each entry further includes information related to a manner of identifying a web site server that responded to the request for that entry, wherein each of the interaction type definitions is associated with a logical site definition that specifies a manner of identifying a web site server related to the web site and that specifies times when that logical site definition is applicable, and wherein the determining that an entry matches an interaction type definition further includes determining that the entry includes information related to the manner of identifying the web site server that matches the manner of identifying a web site server specified by the logical site definition associated with that interaction type definition and that the time of occurrence included in the contained information for the entry is among the interaction times during which that logical site definition is specified to be applicable.

30. The method of claim 28 wherein at least some of the included URLs include a query string portion that is empty or a URL path portion that is empty.

31. The method of claim 28 wherein the associated interaction times for each of the interaction type definitions includes a range of dates.

32. The method of claim 28 including presenting information for at least some of the stored indications.

33. A computer-implemented method for analyzing interaction data to identify occurrences of defined types of interactions by using information about when interactions occur, the method comprising:

receiving an indication of interaction data that is associated with a content set and that has multiple entries, each entry related to an interaction with the content set and containing information that includes an identifier related to when the interaction occurred;

receiving an indication of multiple interaction type definitions that each specify a type of interaction with the content set and have an indication of a version such that the interaction type definition is applicable to interactions related to that version; and

for each entry,

determining whether the entry matches one of the interaction type definitions by containing information about an interaction of the type specified by that one interaction type definition and by containing an identifier that corresponds to the version indicated for that one interaction type definition; and

when it is determined that the entry matches one of the interaction type definitions, indicating an occurrence of that interaction type.

34. The method of claim 33 wherein each identifier is a version identifier, and wherein an identifier is determined to match a version indicated by an interaction type definition if the indicated version is compatible with the version identified by the identifier.

35. The method of claim 33 wherein each identifier is a version identifier, and wherein an identifier is determined to match a version indicated by an interaction type definition if the indicated version is identical to the version identified by the identifier.

36. The method of claim 33 wherein each identifier indicates a time, and wherein an identifier is determined to match a version indicated by an interaction type definition if the indicated version is applicable at the time indicated by the identifier.

37. The method of claim 36 wherein each version has associated effective times that indicate the times at which the version is applicable.

38. The method of claim 33 wherein each identifier contained in an entry is a value of a specified query parameter name from a query string portion of a specified URL for that entry.

39. The method of claim 33 wherein each identifier contained in an entry is a portion of a URL path portion of a specified URL for that entry.

40. A computer-implemented method for analyzing interaction data for a web site to identify occurrences of defined types of interactions using information about times of the occurrences, the method comprising:

receiving an indication of interactions with a web site that each specify a URL, a first of the interactions occurring at a first time and specifying a first URL and a second of the interactions occurring at a second time and also specifying the first URL;

receiving an indication of multiple interaction type definitions that each specify a URL pattern and associated interaction times during which that interaction type definition is applicable to interactions that occur;

determining that the first interaction matches only a first of the interaction type definitions, the matching such that the first URL specified in the first interaction matches the URL pattern of both the first interaction type definition and a second interaction type definition and such that the first time at which the first interaction occurred is among the interaction times during which the first interaction type definition is applicable but is not among the interaction times during which the second interaction type definition is applicable;

determining that the second interaction matches only a second of the interaction type definitions, the matching such that the first URL specified in the second interaction matches the URL pattern of both the first interaction type definition and the second interaction type definition and such that the second time at which the second interaction occurred is among the interaction times during which the second interaction type definition is applicable but is not among the interaction times during which the first interaction type definition is applicable; and

based on the determined matches, providing indications of occurrences of each of the first and second interaction types.

41. The method of claim 40 wherein a third of the interactions occurred at a third time and specified the first URL, and including determining that the third interaction does not match any of the interaction type definitions.

42. The method of claim 40 wherein a third of the interactions occurred at the first time and specified a third URL, and including determining that the third interaction does not match any of the interaction type definitions.

43. The method of claim 40 wherein a third of the interactions occurred at a third time and specified the first URL, and including determining that the third interaction matches multiple of the interaction type definitions.

44. The method of claim 40 wherein a third of the interactions occurred at the first time and specified a third URL, and including determining that the third interaction matches multiple of the interaction type definitions.