System and methods for generating and displaying web site usage data

• US 6,470,383 B1
• Filed: 10/22/1998
• Issued: 10/22/2002
• Est. Priority Date: 10/15/1996
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Mercury Interactive Corp. (HP Inc.)
• Inventors: Weinberg, Amir, Leshem, Eran
• Primary Examiner(s): Harrell, Robert B.
• Assistant Examiner(s): Jaroenchonwanit, Bunjob
• Application Number:
  US09/177,222
• Time in Patent Office: 1,461 Days
• Field of Search: 709/217, 709/223, 709/224, 709/218, 709/238, 707/501, 345/329, 345/349, 345/356, 345/357, 345/440, 345/581, 345/606, 345/706, 345/733, 345/734, 345/736, 345/737, 345/738, 345/783, 345/753
• US Class Current: 709/223
• CPC Class Codes: G06F 11/32 with visual or acoustical indication of the functioning of the machine
• CPC Class Codes: G06F 11/323 Visualisation of programs or trace data
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A visual Web site analysis program, implemented as a collection of software components, provides a variety of features for facilitating the analysis and management of Web sites and Web site content. A mapping component scans a Web site over a network connection and builds a site map which graphically depicts the URLs and links of the site. Site maps are generated using a unique layout and display methodology which allows the user to visualize the overall architecture of the Web site. Various map navigation and URL filtering features are provided to facilitate the task of identifying and repairing common Web site problems, such as links to missing URLs. A dynamic page scan feature enables the user to include dynamically-generated Web pages within the site map by capturing the output of a standard Web browser when a form is submitted by the user, and then automatically resubmitting this output during subsequent mappings of the site. The Web site analysis program is implemented using an extensible architecture which includes an API that allows plug-in applications to manipulate the display of the site map. Various plug-ins are provided which utilize the API to extend the functionality of the analysis program, including an action tracking plug-in which detects user activity and behavioral data (link activity levels, common site entry and exit points, etc.) from server log files and then superimposes such data onto the site map.

Claims

1. A method of facilitating the usage analysis of a web site, the method comprising the computer-implemented steps of:

(a) providing an access log which includes records of accesses by users to content objects of the web site;

(b) scanning the web site to identify at least (i) a plurality of content objects of the web site and (ii) a plurality of user-selectable links between the content objects, the step of scanning comprising generating a data structure that represents the identified content objects and links;

(c) processing the access log to identify a navigation path taken by a user while accessing the web site, the navigation path comprising at least one of the user-selectable links identified in step (b);

(d) repeating step (c) for multiple additional users to generate web site activity data for a plurality of users; and

(e) displaying the web site activity data within a graphical web site map that includes representations of a plurality of the content objects and links identified in step (b), wherein the web site activity data is displayed within the web site map in association with the representations to which the activity data corresponds.

2. The method according to claim 1, wherein step (e) comprises displaying link activity data, the link activity data indicating usage levels of individual links identified in step (b).

3. The method according to claim 1, wherein step (e) comprises displaying exit point data, the exit point data indicating levels of web site exit events from individual content objects identified in step (b), and wherein an access by a visitor to a content object of the web site is treated as an exit event for purposes of generating the exit point data if, based on the access log, the visitor did not subsequently access the web site for a selected period of time.

4. The method according to claim 1, wherein step (e) comprises generating a graphical site map which includes a visual representation of the web site activity data.

5. The method according to claim 1, wherein step (e) comprises superimposing the web site activity data onto the graphical web site map.

6. The method according to claim 5, wherein the step of superimposing comprises making an application program interface call to modify a display attribute of a link object, the link object representing a user-selectable link identified in step (b).

7. The method according to claim 5, wherein the step of superimposing comprises displaying numerical values within the web site map in conjunction with graphical representations of the links, the numerical values indicating respective usage levels of individual links.

8. The method according to claim 5, wherein the step of superimposing comprises displaying different links in different colors to represent different respective ranges of link activity.

9. The method according to claim 8, wherein the step of superimposing further comprises hiding graphical representations of links that have link activity levels falling below a prespecified threshold.

10. The method according to claim 1, wherein step (c) comprises using the access log to identify first and second content objects accessed by the user, and using the data structure generated in step (b) to determine whether a link exists within the web site between the first and second content objects.

11. The method according to claim 10, wherein step (c) further comprises using the access timestamps within the access log to determine an order in which the user accessed the first and second content objects.

12. The method according to claim 1, wherein step (e) comprises displaying a complete navigation path followed by a user of the site.

13. The method according to claim 1, further comprising the computer-implemented step of filtering the access log prior to step (c) based on a filter specified by the user.

14. A computer program for facilitating the analysis of a web site which maintains an access log, comprising, on a computer-readable medium:

a first component which scans the web site and parses documents of the web site to identify at least an organizational structure of content objects and links of the web site, the first component generating a data structure which represents the organizational structure of the content objects and links, and generating a graphical web site map which shows a structural arrangement of the content objects and links; and

a second component which generates web site usage data by at least processing the access log in combination with the data structure to identify links followed by users of the web site, the second component displaying within the graphical web site map the web site usage data in association with the content objects and/or links of the web site to which such usage data corresponds.

15. The computer program of claim 14, wherein the first and second components interoperate to generate a graphical site map which includes visual representations of the web site usage data.

16. The computer program of claim 15, wherein the site map includes graphical representations of the content objects and the links, and the usage data is visually represented within the site map by at least color-coding the graphical representations of the content objects and/or the links.

17. The computer program of claim 15, wherein the first component comprises an application program interface (API) which provides access to the data structure, and the second component uses the API to superimpose the web site usage data onto the site map.

18. The computer program of claim 14, wherein the web site usage data includes link activity data, the link activity data indicating usage levels of individual links of the web site.

19. The computer program of claim 14, wherein the web site usage data includes exit point data, the exit point data indicating levels of web site exit events from individual content objects of the web site, wherein an access by a user to a content object of the web site is treated as an exit event for purposes of generating the exit point data if, based on the access log, the user did not subsequently access the web site for a selected period of time.

20. The computer program of claim 14, wherein the web site usage data comprises complete navigation paths followed by individual users of the site.

21. The computer program of claim 14, wherein the second component identifies a navigation path followed by a user by at least

(a) using the access log to identify first and second content objects accessed by the user; and

(b) using the data structure to determine whether a link to the second content object exists within the first content object.

22. The computer program of claim 14, wherein the second component implements an access log filter which allows a user to filter the access log prior to generation of the web site usage data.

27. The method according to claim 21, wherein step (c) comprises displaying exit point data, the exit point data indicating levels of web site exit events from individual content objects identified in step (a), wherein an access by a user to a content object of the web site is treated as an exit event for purposes of generating the exit point data if, based on the access log, the user did not subsequently access the web site for a selected period of time.

23. A method of facilitating the usage analysis of a web site which maintains an access log, the access log including records of accesses by users to content objects of the web site, the method comprising the computer-implemented steps of:

(a) scanning the web site to identify at least (i) a plurality of content objects of the web site and (ii) a plurality of links between the content objects, the step of scanning comprising generating a data structure which represents an organizational structure of the content objects and the links;

(b) processing the access log in combination with the data structure generated in step (a) to generate web site usage data, the step of generating web site usage data comprising identifying links followed by users while accessing the web site; and

(c) generating, on a display screen, a site map which includes graphical representations of content objects and links identified in step (a), and which includes a superimposed representation of the web site usage data generated in step (b).

24. The method according to claim 23, wherein step (c) comprises displaying link activity data, the link activity data indicating usage levels of individual links identified in step (a).

25. The method according to claim 23, wherein the step of displaying link activity data comprises displaying numerical values within the site map in conjunction with the graphical representations of the links, the numerical values indicating respective usage levels of individual links.

26. The method according to claim 24, wherein the step of displaying link activity data comprises displaying different links in different colors to represent different respective ranges of link activity.

28. The method according to claim 23, wherein step (b) comprises using the access log file to identify first and second content objects accessed by the user, and using the data structure generated in step (a) to determine whether a user-selectable link exists within the web site from the first content object to the second content object.

29. A method of processing an access log of a web site to determine a navigation path followed by a user of the site, the access log including records of accesses by users to content objects of the web site, the method comprising the computer-implemented steps of:

(a) identifying within the access log a first content object that was accessed by the user;

(b) identifying within the access log a second content object of the web site that was accessed by the user prior to accessing the first content object;

(c) determining whether the second content object includes a user-selectable link to the first content object, and when said link exists, recording within a data structure an indication that the user followed the link to navigate from the second content object to the first content object; and

(d) repeating at least steps (a)-(c) for a plurality of users to generate link activity data that indicates usage levels of user-selectable links.

30. The method of claim 29, further comprising the steps of, when no link to the first content object is found in step (c):

(e) identifying within the access log a third content object of the web site that was accessed by the user prior to accessing the second content object; and

(f) determining whether the third content object includes a user-selectable link to the first content object, and when said link within the third content object exists, recording within the data structure an indication that the user followed the link to navigate from the third content object to the first content object.

31. The method of claim 29, further comprising the step of displaying the link activity data on a graphical map of the web site.

32. A method of facilitating the usage analysis of a web site, the method comprising:

maintaining an access log which includes records of accesses by users to nodes of the web site;

generating exit point data reflecting frequencies with which individual nodes of the web site have served as exit points for exiting the web site, wherein generating the exit point data comprises processing the access log to identify, for each of a plurality of users, a last node accessed during a visit to the web site, and wherein generating the exit point data further comprises treating an access by a user to a web site node as an exit event if, based on the access log, the user did not subsequently access the web site for a predetermined period of time; and

displaying the exit point data within a map of the web site.

33. The method of claim 32, wherein the exit point data is displayed within the web site map using graphical representations of the nodes of the web site.

34. The method of claim 33, wherein the graphical representations of the nodes are arranged within the map to show an architecture of the web site.

35. The method of claim 33, wherein the exit point data is displayed by color-coding graphical representations of the nodes.

36. The method of claim 35, further comprising displaying a color-coding key in association with the web site map, the color-coding key indicating assignments of colors to exit point frequency ranges.

37. The method of claim 36, further comprising:

presenting a user with a variable control that allows the user to interactively adjust respective thresholds of at least some of the exit point frequency ranges; and

modifying display colors of the graphical representations of the nodes within the web site map in response to adjustments of the thresholds by the user.

38. The method of claim 32, wherein the exit point data is displayed in different colors such that the different colors represent different frequency ranges of the exit point data.

39. A method of generating usage data that reflects usage of a web site by users, the method comprising:

(a) processing an access log that includes records of accesses by users to nodes of the web site to identify a node accessed by a user during navigation of the web site;

(b) updating a data structure to indicate that the node was used as an exit point from the web site if, based on the access log, the user did not access the web site for a selected period of time after accessing the node; and

(c) repeating (a) and (b) for a plurality of additional users to generate exit point data that reflects frequencies with which specific nodes of the web site have served as web site exit points.

40. The method of claim 39, further comprising displaying the exit point data within a graphical web site map at least in-part by highlighting graphical representations of corresponding nodes of the web site.

41. The method of claim 40, wherein highlighting the graphical representations comprises color-coding the graphical representations.

42. The method of claim 39, wherein updating the data structure comprises incrementing an exits counter for the node.

45. The method of claim 42, wherein the graphical representations of the content objects and links are arranged within the map to show a general architecture of the web site.

43. A method of facilitating the usage analysis of a web site, the method comprising:

accessing an access log that includes records of accesses by users to content objects of the web site;

analyzing the access log in combination with an organizational structure of content objects and links of the web site to determine a complete navigation path followed by a user of the web site, wherein determining the complete navigation path comprises identifying links followed by the user to navigate between content objects of the web site; and

displaying the complete navigation path of the user.

44. The method of claim 43, wherein displaying the navigation path of the user comprises displaying the navigation path within a web site map that includes graphical representations of content objects and links of the web site.

46. The method of claim 43, further comprising displaying, in conjunction with the complete navigation path, an identifier of the user.

47. The method of claim 43, wherein displaying the complete navigation path comprises displaying URL access time tags.

48. A method of facilitating the usage analysis of a web site, the method comprising:

generating a graphical web site map which graphically depicts a structural arrangement of content objects of the web site;

analyzing an access log that reflects accesses by users to content objects of the web site to generate usage data for the web site, the web site usage data reflecting accesses by a plurality of users; and

displaying the usage data within the graphical web site map by modifying graphical representations of the content objects and/or links to which such usage data corresponds.

49. The method of claim 48, wherein the usage data is displayed in association with graphical representations of the content objects.

50. The method of claim 48, wherein the usage data is link activity data.

51. The method of claim 48, wherein the usage data includes a navigational path followed by at least one user of the web site.

52. The method of claim 48, wherein the usage data is exit point data indicative of web site exit events from individual content objects of the web site, wherein an access by a user to a content object of the web site is treated as an exit event for purposes of generating the exit point data if, based on the access log, the visitor did not subsequently access the web site for a selected period of time.

53. The method of claim 48, wherein the usage data is displayed in different colors to represent different respective ranges of user activity.

54. The method of claim 48, wherein the usage data comprises web site entry point data.

55. A method of facilitating the usage analysis of a web site, the method comprising:

analyzing an access log that reflects accesses by users to content objects of the web site to generate link activity data, the link activity data reflecting frequencies with which individual user-selectable links of the web site have been traversed by visitors to the web site; and

displaying the link activity data within a web site map that includes graphical representations of user-accessible content objects and user-selectable links between the content objects, the link activity data being displayed in association with at least one user-selectable link to which the link activity data corresponds.

56. The method of claim 55, wherein displaying the link activity data comprises color coding at least some of the graphical representations of links such that different link colors represent different levels of usage by visitors to the web site.

57. The method of claim 55, further comprising hiding the graphical representations of links that have link activity levels falling below a specified threshold.