Techniques for specifying and collecting data aggregations

• US 7,689,616 B2
• Filed: 04/15/2005
• Issued: 03/30/2010
• Est. Priority Date: 04/15/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Peterson, Glenn R.
• Primary Examiner(s): Boccio; Vincent
• Assistant Examiner(s): FILIPCZYK, MARCIN R
• Application Number:
  US11/107,021
• Publication Number:
  US 20060235879A1
• Time in Patent Office: 1,810 Days
• Field of Search: 707/1, 707/3, 707/6, 707/7, 707/100, 707/101, 707/104.1, 707/200, 707/10
• US Class Current: 707/737
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

Data records containing one or more fields, which can be considered keys and/or values, are received, and processed such that data values of records that contain key values of interest are aggregated together. The keys of the resultant aggregations or “resultant keys” are created under the control of simple parameters to an aggregation framework. Similarly, the particular aggregations performed are also under the control of a similar set of simple parameters to the aggregation framework. Mapping of keys to reduce originality is one of the important features of resultant key creation. Finally, the structure of the parameters used to control aggregation is simple, flexible, and powerful.

Claims

1. A method of controlling data aggregation implemented on a computing device comprising:

receiving an individual data record that includes multiple input fields,

upon receiving the individual data record, selecting a subset of the multiple input fields within the individual data record as a resultant key according to a user defined aggregation parameter set, the user defined aggregation parameter set including;

(i) a key control previously defined by the user to specify one or more of the multiple input fields as key components within the individual data record; and

(ii) an aggregation control previously defined by the user to specify a function or an action to be applied to the specified one or more multiple input fields, the resultant key specified by the user defined aggregation parameter set including;

(i) a concatenation of unmapped, mapped, and composite keys, wherein key values for the unmapped key are unmodified, a mapped key operates to perform a look-up for its corresponding key value, and a new key is created for the composite key by combining other keys with the same composite specification; and

(ii) an identifier of a unique record to which the individual data record contributes data;

computing data aggregations of the multiple input fields according to the resultant key specified by the user defined aggregation parameter set; and

producing collections of aggregations populated as specified by the aggregation parameter set defined by the user, wherein the collections of aggregations are made available to the user.

2. The method of claim 1 wherein the selecting is performed dynamically.

3. The method of claim 1 wherein the computing is based on a record control limit.

4. The method of claim 1 wherein the user defined aggregation parameter set is implemented in a two-dimensional array of Booleans.

5. The method of claim 4 further comprising determining aggregation control subsets by Boolean operations.

6. The method of claim 1 further searching the user defined aggregation parameter set for subsets, marking the subsets, and deferring the marked subsets for processing.

7. The method of claim 6 further comprising computing subsets from the marked subsets.

8. A computer comprising:

a processor;

a memory accessed by and operated on by the processor;

an aggregator framework component accessed by and operated on by the processor, further accessing the memory, configured to receive data records that include multiple input fields and aggregate data in the data records with the multiple input fields using a resultant key in the data records,

wherein the resultant key is a user defined aggregation parameter set, the user defined aggregation parameter set is provided to the aggregation framework component and includes;

(i) a key control previously defined by the user to specify one or more of the multiple input fields as key components within the individual data record; and

(ii) an aggregation control previously defined by the user to specify a function or an action to be applied to the specified one or more multiple input fields; and

the resultant key specified by the user defined aggregation parameter set including (i) a concatenation of unmapped, mapped, and composite keys, wherein key values for the unmapped key are unmodified, a mapped key operates to perform a look-up in a dictionary or other mechanism for its corresponding key value, and a new key is created for the composite key by combining other keys with the same composite specification, and (ii) an identifier of a unique record to which the individual data record contributes data;

computing data aggregations of the multiple input fields according to the resultant key specified by the user defined aggregation parameter set; and

producing collections of aggregations populated as specified by the aggregation parameter set defined by the user, wherein the collections of aggregations are made available to a user.

9. The computer of claim 8 wherein the memory includes aggregators that act on the data records.

10. The computer of claim 8 further comprising a data source that provides the data records.

11. The computer of claim 8 further comprising a data streamer that streams the data records such that the data records are processed individually.

12. The computer of claim 8 further comprising aggregation collections wherein aggregated data is placed.

13. A computer-readable storage media comprising computer-executable instructions that, when executed by a processor, perform acts comprising:

receiving an individual data record that includes multiple input fields comprising key values and data values according to one or more aggregation parameter sets defined by a user, the one or more aggregation parameter sets defined by the user including;

(i) a key control previously defined by the user to specify one or more of the multiple input fields as key components within the individual data record; and

(ii) an aggregation control previously defined by the user to specify a function or an action to be applied to the specified one or more multiple input fields,

the key values being a resultant key including (i) a concatenation of unmapped, mapped, and composite keys, wherein key values for the unmapped key are unmodified, a mapped key operates to perform a look-up for its corresponding key value, and a new key is created for the composite key by combining other keys with the same composite specification, and (ii) an identifier of a unique record to which the individual data record contributes data;

computing data aggregations of the multiple input fields according to the resultant key specified by the user defined aggregation parameter set; and

producing collections of aggregations populated as specified by the aggregation parameter set defined by the user, wherein the collections of aggregations are made available to the user.

14. The computer-readable storage media of claim 13 further comprising:

processing the individual data record through its key values; and

identifying key values of the processed individual data record to a particular aggregation collection which may be made available to the user,

wherein the processing includes applying an aggregation parameter set to the individual data record.

15. The computer-readable storage media of claim 13 further comprising the steps of receiving and processing another data record in a queue of data records.