Providing lightweight multidimensional online data storage for web service usage reporting

• US 8,396,828 B2
• Filed: 09/14/2010
• Issued: 03/12/2013
• Est. Priority Date: 09/14/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ball, Christopher, Polinati, Chinna, Guo, Chenlei, Tiwana, Pravjit
• Primary Examiner(s): Jami, Hares
• Application Number:
  US12/882,139
• Publication Number:
  US 20120066204A1
• Time in Patent Office: 910 Days
• Field of Search: 707/600, 707/602, 707/603, 707/607, 707/736, 707/802, 707/803
• US Class Current: 707/602
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Abstract

Distributed and local processes analyze usage data and transform it into objects including timestamps and dimensions. Objects include a position vector to represent dimension analysis and additional attributes associated with measurements of different types. The objects are stored in a multidimensional database indexed on the vector and timestamp attributes.

Claims

1. A method executed at least in part in a computing device for providing online data storage associated with usage reporting, the method comprising:

causing raw usage data to be transformed and aggregated for extraction measurement data at local components;

causing the measurement data to be uploaded to a multidimensional database;

combining the received measurement data with data processed by other local components;

store raw usage data and measurement data to a distributed file system;

combining the raw usage data and the measurement data at the distributed file system;

mapping the raw usage data to a stream comprising a timestamp, at least one dimension, and at least one measurement;

generating at least one vector based on;

ordering names of the at least one dimension alphabetically and generating a string comprising the names and values of each dimension;

hashing the string;

storing the string and the hash in a table of vectors, wherein components of the vectors are stored in a table serving as an index on a dimension; and

selecting vectors based on matching dimension names and values.

2. The method of claim 1, further comprising:

analyzing the raw usage data stored in the distributed file system;

storing results of the analysis in the distributed file system; and

uploading the results of the analysis to the multidimensional database with the measurement data.

3. The method of claim 2, wherein the analysis is on one of a periodic basis and on-demand basis.

4. The method of claim 2, wherein the analysis is performed by configurable analysis routines.

5. The method of claim 1, wherein transformation and aggregation is on one of a periodic basis and on-demand basis.

6. The method of claim 1, further comprising:

generating the usage data for each operation performed by each local component; and

storing the usage data temporarily on a local storage device.

7. The method of claim 1,

wherein the timestamp indicates expiration time for the stored raw usage data and is treated as a special dimension.

8. The method of claim 1, wherein the measurement data is Quality of Service (QoS) data.

9. A computing device for providing online data storage associated with usage reporting, the computing device comprising:

a memory;

a processor coupled to the memory, the processor executing a distributed application in conjunction with instructions stored in the memory, wherein the application is configured to;

cause raw usage data to be transformed and aggregated for extraction of Quality of Service (QoS) data at local components;

cause the Qos data to be uploaded to a multidimensional database;

combine the received QoS data with data processed by other local components;

store the raw usage data and the QoS data at a distributed file system;

analyze the raw usage data stored in the distributed file system;

combine the raw usage data and the QoS data at a distributed file system;

store results of the analysis in the distributed file system;

upload the results of the analysis to the multidimensional database with the QoS data;

map the raw usage data to a stream comprising a timestamp, at least one dimension, and at least one measurement;

generate at least one vector based on;

ordering names of the at least one dimension alphabetically and generating a string comprising the names and values of each dimension;

hash the string;

store the string and the hash in a table of vectors; and

match new QoS data to existing QoS data through a combination of the vectors and an index employing a hash-assisted lookup.

10. The computing device of claim 9, wherein the raw usage data comprises simple tuples.

11. The computing device of claim 9, wherein each of the components of the vectors are stored in a table serving as the index on a dimension.

12. The computing device of claim 11, wherein the application is further configured to:

select vectors based on matching dimension names and values.

13. The computing device of claim 9, wherein each of the measurement values are stored in a sparse table supporting aggregation types comprising at least one from a set of:

a count, a sum, an average, a standard deviation, a minimum, and a maximum.

14. A method executed at least in part in a computing device for providing online data storage associated with usage reporting, the method comprising:

causing raw usage data to be transformed and aggregated for extraction of Quality of Service (QoS) data at local components;

causing the Qos data to be uploaded to a multidimensional database;

combining the received QoS data with data processed by other local components;

storing the raw usage data and the QoS data at a distributed file system;

analyzing the raw usage data stored in the distributed file system;

combining the raw usage data and the QoS data at a distributed file system;

storing results of the analysis in the distributed file system;

uploading the results of the analysis to the multidimensional database with the QoS data;

mapping the raw usage data to a stream comprising a timestamp, at least one dimension, and at least one measurement;

generating at least one vector based on;

ordering names of the at least one dimension alphabetically and generating a string comprising the names and values of each dimension;

hashing the string;

storing the string and the hash in a table of vectors, wherein components of the vectors are stored in a table serving as an index on a dimension;

matching new QoS data to existing QoS data through a combination of the vectors and the index employing a hash-assisted lookup; and

selecting vectors based on matching dimension names and values.

15. The method of claim 14, wherein the method further comprises:

generating a dynamic view at the time of definition of the stream.

16. The method of claim 15, wherein the method further comprises:

displaying in the dynamic view, source data in original format employing at least one join.

17. The method of claim 15, wherein the method further comprises:

optimizing queries against the multidimensional database by a native query optimizer through the dynamic view.