COLLECTING AND AGGREGATING DATA USING DISTRIBUTED RESOURCES

• US 20090248712A1
• Filed: 03/31/2008
• Published: 10/01/2009
• Est. Priority Date: 03/31/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yuan, Zhongsheng
• Granted Patent: US 7,917,541 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases

Abstract

A hierarchial tree is provided to harvest data from data sources. The hierarchial tree includes multiple tree node entities arranged in multiple levels. In one case, leaf node entities of the hierarchial tree are used to collect data from the data sources. The hierarchial tree includes storage resources for storing the collected data. The hierarchial tree further includes computing resources for aggregating the collecting data in one or more aggregation operations and performing other processing operations. A receiving entity can selectively and independently receive parts of the collected data and aggregated data.

Claims

1. A method for providing data, comprising:

collecting data using a hierarchical tree of tree node entities from a plurality of data sources, to thereby provide collected data, the hierarchical tree including a plurality of levels including a leaf level, tree node entities in the leaf level of the hierarchical tree corresponding to leaf node entities;

storing the collected data using storage resources provided by the hierarchical tree;

aggregating the collected data to form aggregated data using computing resources provided by the hierarchical tree, the storage resources and the computing resources together comprising distributed resources provided by the hierarchical tree; and

selectively providing at least parts of the collected data and the aggregated data to a receiving entity.

2. The method of claim 1, wherein the collected data is made available to the receiving entity independently of the aggregated data.

3. The method of claim 1, wherein the collecting is performed using the leaf node entities.

4. The method of claim 1, wherein the collected data reflects performance of devices associated with the leaf node entities.

5. The method of claim 1, further comprising passing information to at least one target leaf node entity in the hierarchical tree, comprising:

providing the information to an initial node entity in the hierarchical tree; and

passing the information from the initial node entity to said at least one target leaf node entity by successively passing the information through a series of linked tree nodes entities which connect the initial node entity to said at least one target leaf node entity.

6. The method of claim 5, wherein the information comprises policy information that provides instructions regarding the operation of said at least one target leaf node entity.

7. The method of claim 1, wherein the aggregating comprises performing successive aggregation operations at respective levels in the hierarchical tree,

wherein an initial aggregation operation performs aggregation directly on the collected data,

wherein each additional aggregation operation performs aggregation on aggregated data provided by an immediately preceding aggregation operation to yield further aggregated data,

wherein a final aggregation operation yields final aggregated data, and

wherein the receiving entity receives the final aggregated data from a representative of the plurality of tree node entities.

8. The method of claim 1, wherein the aggregating comprises at least one of:

computing a sum of a group of collected data items;

generating a count associated with the group of collected data items;

finding a minimum value in the group of collected data items;

or finding a maximum value in the group of collected data items.

9. The method of claim 1, wherein at least a subset of the tree node entities perform respective principal functions, and wherein the storage resources and computing resources of the hierarchical tree comprise spare resources that are not being utilized by the principal functions.

10. A system for providing data, comprising:

a hierarchical tree comprising a plurality of logically linked tree node entities arranged in a plurality of levels including a leaf level, the tree node entities in the leaf level of the hierarchical tree corresponding to leaf node entities,

each leaf node entity comprising a data collection module configured to provide collected data from at least one data source, and a data store configured to store the collected data,

at least a subset of the tree node entities each comprising an aggregation module configured to aggregate the collected data to form aggregated data.

11. The system of claim 10, wherein at least the leaf node entities comprise production devices for delivering a network-accessible service.

12. The system of claim 10, wherein all the tree node entities in the hierarchical tree comprise production devices for delivering a network-accessible service.

13. The system of claim 10, wherein the tree node entities correspond to a set of devices, wherein at least a subset of the set of devices perform respective principal functions unrelated to the collection and aggregation of data.

14. The system of claim 10, wherein the plurality of levels further comprises at least one non-leaf level of the hierarchical tree.

15. The system of claim 14,

wherein at least one aggregation module at the leaf level of the hierarchical tree performs an aggregation operation directly on the collected data, and

wherein at least one aggregation module in a non-leaf level of the hierarchical tree is configured to receive aggregated data from child node entities and perform a further aggregation operation on the received aggregated data to yield further aggregated data.

16. The system of claim 10, wherein the hierarchical tree includes a plurality of component hierarchical trees.

17. A computer-readable medium for storing computer-readable instructions, the computer-readable instructions providing an agent module when executed by one or more processing devices, the computer-readable instructions comprising:

collection module logic configured to provide collected data from at least one data source for storage in a data store; and

aggregation module logic configured to aggregate the collected data to form aggregated data,

the agent module being configured to selectively provide the collected data and the aggregated data to a receiving entity, and

the agent module being provided on a production device, and the production device being a leaf member of a hierarchical tree of tree node entities.

18. The computer-readable medium of claim 17, wherein the collected data reflects performance of the production device.

19. The computer-readable medium of claim 17, further comprising administrative module logic configured to receive information that is successively passed through the hierarchical tree.

20. The computer-readable medium of claim 17, wherein the agent module is further configured to provide the aggregated data to a logically linked parent node entity of the hierarchical tree.