Variable sampling rates for website visitation analysis

• US 20060274763A1
• Filed: 12/27/2005
• Published: 12/07/2006
• Est. Priority Date: 06/03/2005
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Error, Christopher Reid
• Granted Patent: US 8,578,041 B2
• Time in Patent Office: Days
• US Class Current: 370/400
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A data set containing website traffic data or other data is sampled according to a variable sample rate. A target number of samples per time period is established, and a baseline sample rate is determined. Data items in the data set are sampled according to the baseline sample rate, to obtain a sample set. For time periods where the size of the resulting sample set exceeds the target number of samples, a new sample rate is established and the data items for the time period are resampled. Appropriate sampling capability can thus be provided for website traffic in normal time periods, while maintaining capability for handling spikes and other variations in website traffic as may take place in response to certain periodic or non-periodic events.

Claims

1. A method for sampling a data set comprising a plurality of data items, comprising:

establishing a target number of sample items for a time period;

establishing a first sampling rate;

applying the first sampling rate to data items corresponding to the time period to obtain an first sample set comprising a plurality of sample items; and

responsive to the first sample set containing a number of sample items for the time period substantially different from the target number, performing the steps of;

establishing a second sampling rate; and

obtaining a second sample set using the second sampling rate.

2. The method of claim 1, wherein obtaining a second sample set comprises applying the second sampling rate to the data items corresponding to the time period.

3. The method of claim 1, wherein obtaining a second sample set comprises:

establishing a resampling rate based on the second sampling rate and the first sampling rate; and

applying the resampling rate to the first sample set.

4. The method of claim 3, wherein establishing a resampling rate comprises dividing the second sampling rate by the first sampling rate.

5. The method of claim 1, further comprising:

establishing a tolerance factor;

and wherein the steps of establishing a second sampling rate and obtaining a second sample set using the new sampling rate are performed responsive to the first sample set containing a number of sample items substantially different from the target number plus the tolerance factor.

6. The method of claim 1, wherein establishing a target number comprises establishing a target range;

and wherein the steps of establishing a second sampling rate and obtaining a second sample set using the new sampling rate are performed responsive to the first sample set containing a number of sample items for the time period falling outside the target range.

7. The method of claim 1, further comprising generating a report using at least one of the first and second sample sets.

8. The method of claim 1, wherein the data items describe website visits.

9. The method of claim 8, further comprising generating a website traffic report using at least one of the first and second sample sets.

10. The method of claim 1, wherein the time period comprises a day.

11. The method of claim 1, further comprising:

applying the first sampling rate to data items corresponding to a second time period to obtain an third sample set comprising a plurality of sample items; and

responsive to the third sample set containing a number of sample items for the second time period substantially different from the target number, performing the steps of;

establishing a third sampling rate; and

obtaining a fourth sample set using the third sampling rate.

12. The method of claim 1, further comprising:

applying the second sampling rate to data items corresponding to a second time period to obtain an third sample set comprising a plurality of sample items; and

responsive to the third sample set containing a number of sample items for the second time period substantially different from the target number, performing the steps of;

establishing a third sampling rate; and

obtaining a fourth sample set using the third sampling rate.

13. The method of claim 1, wherein establishing a target number of sample items comprises establishing a target number based on at least one selected from the group consisting of performance, traffic, load factors, storage space, and reporting needs.

14. The method of claim 1, wherein establishing a target number of sample items comprises receiving a target number.

15. The method of claim 1, wherein establishing a first sampling rate comprises establishing a first sampling rate based on the target number of sample items for a time period and an expected number of data items for a time period.

16. The method of claim 1, wherein establishing a first sampling rate comprises receiving a first sampling rate.

17. A method for sampling a data set comprising a plurality of data items, each data item associated with a time period, comprising:

establishing a target number of sample items per time period;

establishing a first sampling rate;

applying the first sampling rate to data items corresponding to each time period to obtain, for each time period, a first sample set comprising a plurality of sample items for the time period; and

responsive to the first sample set for a time period containing a number of sample items for the time period substantially different from the target number, performing the steps of;

establishing a second sampling rate for the time period; and

obtaining a second sample set for the time period using the second sampling rate.

18. A computer program product for sampling a data set comprising a plurality of data items, comprising:

a computer-readable medium; and

computer program code, encoded on the medium, for;

establishing a target number of sample items for a time period;

establishing a first sampling rate;

applying the first sampling rate to data items corresponding to the time period to obtain an first sample set comprising a plurality of sample items; and

responsive to the first sample set containing a number of sample items for the time period substantially different from the target number, performing the steps of;

establishing a second sampling rate; and

obtaining a second sample set using the second sampling rate.

19. The computer program product of claim 18, wherein the computer program code for obtaining a second sample set comprises computer program code for applying the second sampling rate to the data items corresponding to the time period.

20. The computer program product of claim 18, wherein the computer program code for obtaining a second sample set comprises computer program code for:

establishing a resampling rate based on the second sampling rate and the first sampling rate; and

applying the resampling rate to the first sample set.

21. The computer program product of claim 20, wherein the computer program code for establishing a resampling rate comprises computer program code for dividing the second sampling rate by the first sampling rate.

22. The computer program product of claim 18, further comprising computer program code for:

establishing a tolerance factor;

and wherein the computer program code for establishing a second sampling rate and obtaining a second sample set using the new sampling rate are invoked responsive to the first sample set containing a number of sample items substantially different from the target number plus the tolerance factor.

23. The computer program product of claim 18, wherein the computer program code for establishing a target number comprises computer program code for establishing a target range;

and wherein the computer program code for establishing a second sampling rate and obtaining a second sample set using the new sampling rate are invoked responsive to the first sample set containing a number of sample items for the time period falling outside the target range.

24. The computer program product of claim 18, further comprising computer program code for generating a report using at least one of the first and second sample sets.

25. The computer program product of claim 18, wherein the data items describe website visits.

26. The computer program product of claim 25, further comprising computer program code for generating a website traffic report using at least one of the first and second sample sets.

27. The computer program product of claim 18, wherein the time period comprises a day.

28. The computer program product of claim 18, further comprising computer program code for:

applying the first sampling rate to data items corresponding to a second time period to obtain an third sample set comprising a plurality of sample items; and

responsive to the third sample set containing a number of sample items for the second time period substantially different from the target number;

establishing a third sampling rate; and

obtaining a fourth sample set using the third sampling rate.

29. The computer program product of claim 18, further comprising computer program code for:

applying the second sampling rate to data items corresponding to a second time period to obtain an third sample set comprising a plurality of sample items; and

responsive to the third sample set containing a number of sample items for the second time period substantially different from the target number;

establishing a third sampling rate; and

obtaining a fourth sample set using the third sampling rate.

30. The computer program product of claim 18, wherein the computer program code for establishing a target number of sample items comprises computer program code for establishing a target number based on at least one selected from the group consisting of performance, traffic, load factors, storage space, and reporting needs.

31. The computer program product of claim 18, wherein the computer program code for establishing a target number of sample items comprises computer program code for receiving a target number.

32. The computer program product of claim 18, wherein the computer program code for establishing a first sampling rate comprises computer program code for establishing a first sampling rate based on the target number of sample items for a time period and an expected number of data items for a time period.

33. The computer program product of claim 18, wherein the computer program code for establishing a first sampling rate comprises computer program code for receiving a first sampling rate.

34. A system for sampling a data set comprising a plurality of data items, comprising:

a log processing module, for;

establishing a target number of sample items for a time period, establishing a first sampling rate;

applying the first sampling rate to data items corresponding to the time period to obtain an first sample set comprising a plurality of sample items; and

responsive to the first sample set containing a number of sample items for the time period substantially different from the target number, performing the steps of;

establishing a second sampling rate; and

obtaining a second sample set using the second sampling rate; and

a storage device, for storing at least one of the first and second sample sets.

35. The system of claim 34, wherein the log processing module obtains a second sample set by applying the second sampling rate to the data items corresponding to the time period.

36. The system of claim 34, wherein the log processing module obtains a second sample set by:

establishing a resampling rate based on the second sampling rate and the first sampling rate; and

applying the resampling rate to the first sample set.

37. The system of claim 36, wherein the log processing module establishes a resampling rate by dividing the second sampling rate by the first sampling rate.

38. The system of claim 34, wherein:

the log processing module establishes a tolerance factor;

and wherein the the log processing module establishes a second sampling rate and obtains a second sample set using the new sampling rate responsive to the first sample set containing a number of sample items substantially different from the target number plus the tolerance factor.

39. The system of claim 34, wherein:

the log processing module establishes a target number by establishing a target range;

and wherein the log processing module establishes a second sampling rate and obtains a second sample set using the new sampling rate responsive to the first sample set containing a number of sample items for the time period falling outside the target range.

40. The system of claim 34, further comprising a report generator, for generating a report using at least one of the first and second sample sets.

41. The system of claim 34, wherein the data items describe website visits.

42. The system of claim 41, further comprising a report generator, for generating a website traffic report using at least one of the first and second sample sets.

43. The system of claim 34, wherein the time period comprises a day.

44. The system of claim 34, wherein:

the log processing module applies the first sampling rate to data items corresponding to a second time period to obtain an third sample set comprising a plurality of sample items; and

responsive to the third sample set containing a number of sample items for the second time period substantially different from the target number, the log processing module;

establishes a third sampling rate; and

obtains a fourth sample set using the third sampling rate.

45. The system of claim 34, wherein:

the log processing module applies the second sampling rate to data items corresponding to a second time period to obtain an third sample set comprising a plurality of sample items; and

responsive to the third sample set containing a number of sample items for the second time period substantially different from the target number, the log processing module;

establishes a third sampling rate; and

obtains a fourth sample set using the third sampling rate.

46. The system of claim 34, wherein the log processing module establishes a target number of sample items by establishing a target number based on at least one selected from the group consisting of performance, traffic, load factors, storage space, and reporting needs.

47. The system of claim 34, wherein the log processing module establishes a target number of sample items by receiving a target number.

48. The system of claim 34, wherein the log processing module establishes a first sampling rate by establishing a first sampling rate based on the target number of sample items for a time period and an expected number of data items for a time period.

49. The system of claim 34, wherein the log processing module establishes a first sampling rate by receiving a first sampling rate.