Dynamic Data Aggregation from a Plurality of Data Sources

• US 20120130987A1
• Filed: 11/19/2010
• Published: 05/24/2012
• Est. Priority Date: 11/19/2010
• Status: Active Grant

• Current Assignee: Daedalus Blue LLC
• Original Assignee: International Business Machines Corporation
• Inventors: Bose, Abhijit, Smadi, Mithkal M., Sun, Jimeng, Velpuri, Chandra Kumar
• Granted Patent: US 9,292,575 B2
• Time in Patent Office: Days
• US Class Current: 707/718
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination

Abstract

A method for dynamically aggregating data is provided. A server device receives a subscriber request for a report based on a subset of metadata contained in a data dimensions catalog. The server device analyzes data aggregation requirements from a plurality of data sources for the report based on the subset of metadata defined in the subscriber request. The server device generates a data access plan for movement of data from the plurality of data sources based on the data aggregation requirements for the report. Then, the server device executes the data access plan to fetch the data from the plurality of data sources based on the data aggregation requirements for the report. A computer system and computer program product for dynamically aggregating data are also provided.

Claims

1. A computer implemented method for dynamically aggregating data, the computer implemented method comprising:

receiving, by a server device, a subscriber request for a report based on a subset of metadata contained in a data dimensions catalog;

analyzing, by the server device, data aggregation requirements from a plurality of data sources for the report based on the subset of metadata defined in the subscriber request;

generating, by the server device, a data access plan for movement of data from the plurality of data sources based on the data aggregation requirements for the report; and

executing, by the server device, the data access plan to fetch the data from the plurality of data sources based on the data aggregation requirements for the report.

2. The computer implemented method of claim 1 further comprising:

sending, by the server device, the data fetched from the plurality of data sources based on the data aggregation requirements for the report to a client application.

3. The computer implemented method of claim 1 further comprising:

receiving, by the server device, metadata from each of the plurality of data sources describing dimensions of data stored in each of the plurality of data sources; and

aggregating, by the server device, the metadata from each of the plurality of data sources to build the data dimensions catalog of all available data dimensions across the plurality of data sources.

4. The computer implemented method of claim 1 further comprising:

associating a set of metadata to a plurality of data dimensions stored in a data source.

5. The computer implemented method of claim 1 further comprising:

receiving an updated set of metadata from a data source that reflects a data dimensions update in the data source.

6. The computer implemented method of claim 1 wherein the server device sends the data access plan to the plurality of data sources along with client connection information in order for the plurality of data sources to send the requested data directly to a client application instead of to the server device.

7. The computer implemented method of claim 1 wherein the server device provides a distributed data aggregation service to subscribing client applications without utilizing a centralized multidimensional data warehouse.

8. The computer implemented method of claim 1 wherein metadata includes annotations, and wherein the annotations specify a set of rules about how data dimensions can be aggregated.

9. The computer implemented method of claim 1 wherein the plurality of data sources is a plurality of distributed remote databases for an enterprise.

10. The computer implemented method of claim 2 wherein the client application resides in a client device and communicates with the data aggregation server via an application programming interface, and wherein the application programming interface provides a real-time feed to and from the server device in order to subscribe to or search for available data dimensions.

11. The computer implemented method of claim 1 wherein the data dimensions catalog includes a global metadata tree structure.

12. A computer system for dynamically aggregating data, the computer system comprising:

a central processing unit, a computer-readable memory, and a computer-readable, tangible storage device;

program instructions, stored on the storage device for execution by the central processing unit via the memory, to receive a subscriber request for a report based on a subset of metadata contained in the data dimensions catalog;

program instructions, stored on the storage device for execution by the central processing unit via the memory, to analyze data aggregation requirements from a plurality of data sources for the report based on the subset of metadata defined in the subscriber request;

program instructions, stored on the storage device for execution by the central processing unit via the memory, to generate a data access plan for movement of data from the plurality of data sources based on the data aggregation requirements for the report; and

program instructions, stored on the storage device for execution by the central processing unit via the memory, to execute the data access plan to fetch the data from the plurality of data sources based on the data aggregation requirements for the report.

13. A computer program product comprising a computer-readable, tangible storage device(s) and computer-readable program instructions stored on the computer-readable, tangible storage device(s) to dynamically aggregating data, the computer-readable program instructions, when executed by a central processing unit:

receive a subscriber request for a report based on a subset of metadata contained in a data dimensions catalog;

analyze data aggregation requirements from a plurality of data sources for the report based on the subset of metadata defined in the subscriber request;

generate a data access plan for movement of data from the plurality of data sources based on the data aggregation requirements for the report; and

execute the data access plan to fetch the data from the plurality of data sources based on the data aggregation requirements for the report.

14. The computer program product of claim 13 further comprising computer-readable program instructions, which are stored on the computer-readable, tangible storage device(s) and when executed by a central processing unit:

send the data fetched from the plurality of data sources based on the data aggregation requirements for the report to a client application.

15. The computer program product of claim 13 further comprising computer-readable program instructions, which are stored on the computer-readable, tangible storage device(s) and when executed by a central processing unit:

receive metadata from each of the plurality of data sources describing dimensions of data stored in each of the plurality of data sources; and

aggregate the metadata from each of the plurality of data sources to build the data dimensions catalog of all available data dimensions across the plurality of data sources.

16. The computer program product of claim 13 further comprising computer-readable program instructions, which are stored on the computer-readable, tangible storage device(s) and when executed by a central processing unit:

associate a set of metadata to a plurality of data dimensions stored in a data source.

17. The computer program product of claim 13 further comprising computer-readable program instructions, which are stored on the computer-readable, tangible storage device(s) and when executed by a central processing unit:

receive an updated set of metadata from a data source that reflects a data dimensions update in the data source.

18. The computer program product of claim 13 wherein a server device sends the data access plan to the plurality of data sources along with client connection information in order for the plurality of data sources to send the requested data directly to a client application instead of to the server device.

19. The computer program product of claim 13 wherein a server device provides a distributed data aggregation service to subscribing client applications without utilizing a centralized multidimensional data warehouse.

20. The computer program product of claim 13 wherein metadata includes annotations, and wherein the annotations specify a set of rules about how data dimensions can be aggregated.