Performing flexible pivot querying of monitoring data using a multi-tenant monitoring system

• US 9,043,327 B1
• Filed: 06/13/2013
• Issued: 05/26/2015
• Est. Priority Date: 06/13/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Heitmann, John W., Dunagan, John David, Pai, Derek Solomon, Shim, Caitlyn Rose, Foster, Michael R.
• Primary Examiner(s): Lee, Wilson
• Application Number:
  US13/917,605
• Time in Patent Office: 712 Days
• Field of Search: None
• US Class Current: 707/737
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Pivot queries can be performed using stored monitoring data. Pivot queries can be received. Monitoring data can be read from data files. Each data file can store monitoring data for a plurality of tenants. The read monitoring data can be aggregated based on metric types across various dimensions, according to the pivot queries. Results can be presented in various formats, such as charts and graphs. Reading, aggregating, and presenting results can be performed within a multi-tenant monitoring system.

Claims

1. A method, implemented at least in part by a multi-tenant monitoring system comprising one or more computing devices, for performing pivot queries using monitoring data, the method comprising:

receiving, by the multi-tenant monitoring system, the pivot query from a tenant, wherein the pivot query indicates one or more metric types and one or more dimensions;

reading, by the multi-tenant monitoring system, monitoring data associated with the tenant from one or more data files, wherein the monitoring data stored in the one or more data files is not aggregated across dimensions, and wherein each of the one or more data files supports storing monitoring data for a plurality of tenants;

aggregating, by the multi-tenant monitoring system, the monitoring data read from the one or more data files, wherein the aggregating comprises aggregating at least one of the one or more metric types across at least one of the one or more dimensions; and

based at least in part on the aggregated monitoring data, outputting results of the pivot query.

2. The method of claim 1 further comprising:

for each of a plurality of pivot queries;

receiving the pivot query from the tenant; and

performing the reading, the aggregating, and the outputting results according to the received pivot query, the one or more metric types, and the one or more dimensions.

3. The method of claim 1 wherein the monitoring data is grouped together for the tenant within the one or more data files.

4. The method of claim 1 wherein the monitoring data stored within the plurality of data files is not indexed at a level of individual observations.

5. The method of claim 1 wherein reading the monitoring data associated with the tenant comprises:

for each time period of a plurality of time periods;

reading from a set of data files storing monitoring data for the time period;

wherein each time period stores monitoring data in a different set of data files.

6. The method of claim 1 wherein aggregating the monitoring data comprises:

aggregating the monitoring data across one or more dimensions.

7. The method of claim 1 wherein the pivot query is performed in linear time.

8. The method of claim 1 wherein aggregating the monitoring data is performed in a single pass for one or more metric types across one or more dimensions.

9. The method of claim 1 wherein performing the pivot query scales linearly when reading the monitoring data for each of a plurality of time periods and corresponding sets of data files.

10. The method of claim 1 further comprising:

performing the method for each of a plurality of pivot queries;

comparing the results from the plurality of pivot queries; and

making one or more recommendations based on the comparing.

11. The method of claim 1 further comprising:

for each of a plurality of pivot queries;

receiving the pivot query from the tenant; and

performing the reading, the aggregating, and the outputting results according to the received pivot query, the one or more metric types, and the one or more dimensions;

comparing the results of the plurality of pivot queries; and

making one or more recommendations based on the comparing.

12. A multi-tenant monitoring system comprising one or more computing devices for serving pivot queries using monitoring data, the multi-tenant monitoring system comprising:

one or more systems comprising processing units, memory, and file storage;

the one or more systems configured to perform operations for serving pivot queries using monitoring data, the operations comprising;

receiving the pivot query from a tenant;

reading monitoring data associated with the tenant from one or more data files, wherein the monitoring data stored in the one or more data files is not aggregated across dimensions, and wherein each of the one or more data files supports storing monitoring data for a plurality of tenants; and

providing results of reading the monitoring data for aggregation to support the pivot query.

13. The multi-tenant monitoring system of claim 12 wherein the pivot query indicates one or more metric types and one or more dimensions.

14. The multi-tenant monitoring system of claim 12 further comprising:

based at least in part on the results of reading the monitoring data, aggregating the monitoring data read from the one or more data files; and

based at least in part on the aggregated monitoring data, outputting results of the pivot query.

15. The multi-tenant monitoring system of claim 14 the operations further comprising:

for each of a plurality of pivot queries;

receiving the pivot query from the tenant, wherein the pivot query indicates one or more metric types and one or more dimensions; and

performing the reading, the aggregating, and the presenting results according to the received pivot query, the one or more metric types, and the one or more dimensions.

16. The multi-tenant monitoring system of claim 14 wherein the pivot query is performed in linear time, and wherein aggregating the monitoring data is performed in a single pass for one or more metric types across one or more dimensions.

17. The multi-tenant monitoring system of claim 14 further comprising:

performing the method for each of a plurality of pivot queries;

comparing the results from the plurality of pivot queries; and

making one or more recommendations based on the comparing.

18. The multi-tenant monitoring system of claim 12 wherein the monitoring data is grouped together for the tenant within the one or more data files, and wherein the monitoring data stored within the plurality of data files is not indexed at a level of individual observations.

19. The multi-tenant monitoring system of claim 12 wherein reading the monitoring data associated with the tenant comprises:

for each time period of a plurality of time periods;

reading from a set of data files storing monitoring data for the time period;

wherein each time period stores monitoring data in a different set of data files.

20. The multi-tenant monitoring system of claim 12 wherein performing the pivot query scales linearly when reading the monitoring data for each of a plurality of time periods and corresponding sets of data files.

21. The multi-tenant monitoring system of claim 12 wherein the results of reading the monitoring data are provided to a different computing device for aggregation.

22. A computer-readable storage medium storing computer-executable instructions for causing one or more computing devices to perform a method for serving pivot queries using monitoring data within a multi-tenant monitoring system, the method comprising:

receiving the pivot query from a tenant, wherein the pivot query indicates at least one metric type and at least one dimensions;

reading monitoring data associated with the tenant from one or more data files, wherein the monitoring data stored in the one or more data files is not aggregated across dimensions, wherein the monitoring data is grouped together for the tenant within the one or more data files, and wherein each of the one or more data files supports storing monitoring data for a plurality of tenants;

aggregating the monitoring data read from the one or more data files; and

based at least in part on the aggregated monitoring data, outputting results of the pivot query.

23. The computer-readable storage medium of claim 22 wherein the monitoring data stored within the plurality of data files is not indexed at a level of individual observations.

24. The computer-readable storage medium of claim 22 wherein the aggregating comprises aggregating the at least one metric type across the at least one dimension.

25. The computer-readable storage medium of claim 22 wherein reading the monitoring data associated with the tenant comprises:

for each time period of a plurality of time periods;

reading from a set of data files storing monitoring data for the time period;

wherein each time period stores monitoring data in a different set of data files.

26. The computer-readable storage medium of claim 22 wherein performing the pivot query scales linearly when reading the monitoring data for each of a plurality of time periods and corresponding sets of data files.

27. The computer-readable storage medium of claim 22 further comprising:

performing the method for each of a plurality of pivot queries;

comparing the results from the plurality of pivot queries; and

making one or more recommendations based on the comparing.

28. A method, implemented at least in part by a multi-tenant monitoring system comprising one or more computing devices, for performing pivot queries using monitoring data, the method comprising:

performing each of a plurality of a plurality of pivot queries, comprising;

receiving, by the multi-tenant monitoring system, the pivot query from a tenant, wherein the pivot query indicates one or more metric types and one or more dimensions;

reading, by the multi-tenant monitoring system, monitoring data associated with the tenant from one or more data files, wherein the monitoring data stored in the one or more data files is not aggregated across dimensions, and wherein each of the one or more data files supports storing monitoring data for a plurality of tenants; and

aggregating, by the multi-tenant monitoring system, the monitoring data read from the one or more data files according to the one or more metric types and the one or more dimensions;

comparing results from performing each of the plurality of pivot queries; and

making one or more recommendations based on the comparing.

29. The method of claim 28 wherein the comparing results comprises comparing a plurality of software components, and wherein the one or more recommendations comprise a recommendation to use one of the plurality of software components instead of a different one of the plurality of software components.