Data format for website traffic statistics

• US 20060277197A1
• Filed: 11/14/2005
• Published: 12/07/2006
• Est. Priority Date: 06/03/2005
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Bailey, Michael Paul
• Granted Patent: US 8,538,969 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Abstract

A data format is optimized for storing data such as website traffic data. The data format enables easy access to and filtering of data, for example in generating website traffic reports. The data format also provides significant data compression. A method for generating a data file according to the data format employs linear compression and indexing to efficiently store the data. Data stored according to the format can be easily retrieved, particularly when a known value is specified and particular entries matching the known value are sought.

Claims

1. A method for storing data, comprising:

receiving a raw data set comprising a plurality of rows of data, each row comprising at least one value for at least one field;

for each field, identifying at least one value present in the raw data set;

for each identified value;

generating a list specifying at least one row containing the value; and

storing a representation of the list.

2. The method of claim 1, wherein storing the representation of the list comprises storing the list.

3. The method of claim 1, further comprising, for each identified value:

prior to storing the representation of the list, applying linear compression to the list;

and wherein storing the representation of the list comprises storing the linearly compressed list.

4. The method of claim 3, wherein, for at least one identified value:

applying linear compression to the list comprises generating start and end row numbers for at least one contiguous range of rows containing the value; and

storing the linearly compressed list comprises storing the start and end row numbers.

5. The method of claim 3, wherein, for at least one identified value:

applying linear compression to the list comprises;

generating start and end row numbers for at least one contiguous range of rows containing the value; and

converting at least one end row number to an offset based on a corresponding start row number; and

storing the linearly compressed list comprises storing each start row number and offset.

6. The method of claim 3, wherein, for at least one identified value:

applying linear compression to the list comprises;

generating start and end row numbers for at least one contiguous range of rows containing the value; and

converting at least one start row number to a start offset based on a previous row number; and

converting each end row number to a start offset based on a corresponding start row number; and

storing the linearly compressed list comprises storing the start and end offsets.

7. The method of claim 6, wherein converting at least one start row number to a start offset based on a previous row number comprises converting at least one start row number to a start offset based on a previous start row number.

8. The method of claim 6, wherein converting at least one start row number to a start offset based on a previous row number comprises converting at least one start row number to a start offset based on a previous end row number.

9. The method of claim 6, wherein storing the representation of the list comprises storing at least one absolute row number, and wherein converting at least one start row number to a start offset based on a previous row number comprises converting at least one start row number to a start offset based on an absolute row number.

10. The method of claim 6, wherein storing the representation of the list comprises storing at least one absolute row number, and wherein converting at least one start row number to a start offset based on a previous row number comprises converting at least one start row number to a start offset based on the most recently stored absolute row number.

11. The method of claim 1, wherein receiving the raw data set comprises receiving a raw data set describing website traffic.

12. The method of claim 11, wherein receiving the raw data set comprises receiving a raw data set describing at least one selected from the group consisting of visitor data, visit data, hits data, and commerce data.

13. The data format of claim 1, wherein receiving the raw data set comprises receiving sampled data.

14. A data format, comprising:

a computer-readable medium, comprising computer-readable data describing;

at least one data value for at least one field; and

for each data value, a representation of a list specifying at least one row containing the value.

15. The data format of claim 14, wherein the representation of the list comprises the list.

16. The data format of claim 14, further the representation of the list comprises a linearly compressed list.

17. The data format of claim 16, wherein the linearly compressed list comprises start and end row numbers for at least one contiguous range of rows containing a value.

18. The data format of claim 16, wherein the linearly compressed list comprises a start row number and an offset for at least one contiguous range of rows containing a value.

19. The data format of claim 16, wherein the linearly compressed list comprises an offset representing a row with respect to a previous row.

20. The data format of claim 14, wherein the computer-readable data describes website traffic.

21. The data format of claim 20, wherein the computer-readable data describes at least one selected from the group consisting of visitor data, visit data, hits data, and commerce data.

22. The data format of claim 14, wherein the computer-readable data is sampled data.

23. A data format, comprising:

a computer-readable medium, comprising computer-readable data describing;

a header, specifying a number of tables;

a table definition block, the table definition block comprising, for each of at least one table, a field list and a table size attribute; and

at least one table, each table comprising;

at least one data value for at least one field; and

for each data value, a representation of a list specifying at least one row containing the value.

24. The data format of claim 23, wherein each table describes website traffic.

25. The data format of claim 24, wherein each table describes at least one selected from the group consisting of visitor data, visit data, hits data, and commerce data.

26. A method for reading data stored in a format comprising at least one data value for at least one field, and for each data value, a representation of a list specifying at least one row containing the value, the method comprising:

retrieving, for at least one data value, the representation of the list containing the value; and

determining, from the representation of the list, a set of at least one row containing the value;

generating output based on the determined set of at least one row.

27. The method of claim 26, wherein the representation of the list comprises a linearly compressed representation of the list, and wherein determining, from the representation of the list, a set of at least one row containing the value comprises decompressing the linearly compressed representation of the list.

28. The method of claim 27, wherein decompressing the linearly compressed representation of the list comprises reading start and end row numbers for at least one contiguous range of rows containing a value, and converting the read start and end row numbers to a list of row numbers.

29. The method of claim 27, wherein decompressing the linearly compressed representation of the list comprises reading a start row number and an offset for at least one contiguous range of rows containing a value, and converting the read start row number and offset to a list of row numbers.

30. The method of claim 26, wherein the data describes website traffic, and wherein generating output based on the determined set of at least one row comprises generating a website traffic report.

31. The method of claim 30, wherein the data describes at least one selected from the group consisting of visitor data, visit data, hits data, and commerce data.

32. A computer program product for storing data, comprising:

a computer-readable medium; and

computer program code, encoded on the medium, for;

receiving a raw data set comprising a plurality of rows of data, each row comprising at least one value for at least one field;

for each field, identifying at least one value present in the raw data set;

for each identified value;

generating a list specifying at least one row containing the value; and

storing a representation of the list.

33. The computer program product of claim 32, further comprising computer program code for, for each identified value:

prior to storing the representation of the list, applying linear compression to the list;

and wherein the computer program code for storing the representation of the list comprises computer program code for storing the linearly compressed list.

34. The computer program product of claim 32, wherein the computer program code for receiving the raw data set comprises computer program code for receiving a raw data set describing website traffic.

35. A computer program product for reading data stored in a format comprising at least one data value for at least one field, and for each data value, a representation of a list specifying at least one row containing the value, the computer program product comprising:

a computer-readable medium; and

computer program code, encoded on the medium, for;

retrieving, for at least one data value, the representation of the list containing the value; and

determining, from the representation of the list, a set of at least one row containing the value;

generating output based on the determined set of at least one row.

36. The computer program product of claim 35, wherein the representation of the list comprises a linearly compressed representation of the list, and wherein the computer program code for determining, from the representation of the list, a set of at least one row containing the value comprises computer program code for decompressing the linearly compressed representation of the list.

37. The computer program product of claim 35, wherein the data describes website traffic, and wherein the computer program code for generating output based on the determined set of at least one row comprises computer program code for generating a website traffic report.

38. A method for storing a series of values, comprising:

storing a series start value; and

storing a length.

39. The method of claim 38, wherein the series start value comprises an offset with respect to a previously stored value.

40. The method of claim 38, wherein the series comprises a plurality of values having a regular pattern.

41. The method of claim 40, further comprising storing an indicator specifying the pattern from a plurality of available patterns.

42. The method of claim 38, wherein the series comprises a plurality of consecutive integers.

43. A method for reading an encoded representation of a series of values, comprising:

reading a series start value;

reading a length; and

generating the series of values from the start value and length.

44. The method of claim 43, wherein the series start value comprises an offset with respect to a previously stored value.

45. The method of claim 43, wherein the series comprises a plurality of values having a regular pattern.

46. The method of claim 45, further comprising reading an indicator specifying the pattern from a plurality of available patterns.

47. The method of claim 43, wherein the series comprises a plurality of consecutive integers.

48. A data format for storing a series of values, comprising:

a series start value; and

a length.

49. The data format of claim 48, wherein the series start value comprises an offset with respect to a previously stored value.

50. The data format of claim 48, wherein the series comprises a plurality of values having a regular pattern.

51. The data format of claim 50, further comprising an indicator specifying the pattern from a plurality of available patterns.

52. The data format of claim 48, wherein the series comprises a plurality of consecutive integers.

53. A computer program product for storing a series of values, comprising:

a computer-readable medium; and

computer program code, encoded on the medium, for;

storing a series start value; and

storing a length.

54. The computer program product of claim 53, wherein the series start value comprises an offset with respect to a previously stored value.

55. The computer program product of claim 53, wherein the series comprises a plurality of values having a regular pattern.

56. The computer program product of claim 55, further comprising computer program code for storing an indicator specifying the pattern from a plurality of available patterns.

57. The computer program product of claim 53, wherein the series comprises a plurality of consecutive integers.

58. A computer program product for reading an encoded representation of a series of values, comprising:

a computer-readable medium; and

computer program code, encoded on the medium, for;

reading a series start value;

reading a length; and

generating the series of values from the start value and length.

59. The computer program product of claim 58, wherein the series start value comprises an offset with respect to a previously stored value.

60. The computer program product of claim 58, wherein the series comprises a plurality of values having a regular pattern.

61. The computer program product of claim 60, further comprising computer program code for reading an indicator specifying the pattern from a plurality of available patterns.

62. The computer program product of claim 58, wherein the series comprises a plurality of consecutive integers.