Data management for data aggregation

• US 8,856,187 B2
• Filed: 02/07/2008
• Issued: 10/07/2014
• Est. Priority Date: 02/07/2008
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated (Kyndryl Holdings, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Kanamatareddy, Ravi K. R.
• Primary Examiner(s): PYO, MONICA M
• Application Number:
  US12/027,284
• Publication Number:
  US 20090204565A1
• Time in Patent Office: 2,434 Days
• Field of Search: 707/999.001, 707/796, 707/802
• US Class Current: 707/802
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Abstract

The invention provides a method, system, and program product for managing data for data aggregation, including data mining and reporting. Locations of a plurality of data to be collected are determined within a source database. Data are simultaneously collected from the plurality of locations and aggregated. The aggregated data are normalized by adding an encryption key and the normalized data are stored. Data at each of the plurality of locations are then released in the source database.

Claims

1. A method of managing data for data aggregation, the method comprising:

determining a plurality of locations of data to be collected within a source database;

acquiring at least one access configuration log of the plurality of locations from which data will be collected;

simultaneously collecting data from the plurality of the locations;

aggregating the collected data;

normalizing the aggregated data, wherein normalizing includes adding an encryption key to the aggregated data;

storing the normalized data; and

releasing the data at each of the plurality of locations in the source database.

2. The method of claim 1, wherein simultaneously collecting data includes buffering the collection based on at least one of the following:

a previous collection;

a current collection; and

an upcoming collection.

3. The method of claim 1, wherein aggregating includes constructing a comma separated value (CSV) data stream from the collected data.

4. The method of claim 1, wherein aggregating includes at least one update selected from a group consisting of:

overwriting old data in a previous collection and inserting new data in a previous collection.

5. The method of claim 1, wherein normalizing further includes at least one action selected from a group consisting of:

compressing the aggregated data; and

converting the aggregated data to another format.

6. The method of claim 1, wherein storing includes determining a size of the normalized data to be stored.

7. A system for managing data for data aggregation, the system comprising:

at least one computing device;

a system for determining a plurality of locations of data to be collected within a source database;

a system for acquiring at least one access configuration log of the plurality of locations from which data will be collected;

a system for simultaneously collecting data from the plurality of the locations;

a system for aggregating the collected data;

a system for normalizing the aggregated data, wherein normalizing includes adding an encryption key to the aggregated data;

a system for storing the normalized data; and

a system for releasing the data at each of the plurality of locations in the source database.

8. The system of claim 7, wherein the system for simultaneously collecting data includes a system for buffering the collection based on at least one of the following:

a previous collection;

a current collection; and

an upcoming collection.

9. The system of claim 7, wherein the system for aggregating includes a system for constructing a comma separated value (CSV) data stream from the collected data.

10. The system of claim 7, wherein the system for aggregating is operable to perform at least one of the following actions:

overwrite old data in a previous collection and insert new data in a previous collection.

11. The system of claim 7, wherein the system for normalizing is operable to perform at least one of the following actions:

compress the aggregated data; and

convert the aggregated data to another format.

12. The system of claim 7, wherein the system for storing includes a system for determining a size of the normalized data to be stored.

13. A program product stored on a computer-readable storage medium, which when executed, manages data for data aggregation, the program product comprising:

program code for determining a plurality of locations of data to be collected within a source database;

program code for acquiring at least one access configuration log of the plurality of locations from which data will be collected;

program code for simultaneously collecting data from the plurality of the locations;

program code for aggregating the collected data;

program code for normalizing the aggregated data, wherein normalizing includes adding an encryption key to the aggregated data;

program code for storing the normalized data; and

program code for releasing the data at each of the plurality of locations in the source database.

14. The program product of claim 13, wherein the program code for simultaneously collecting data includes program code for buffering the collection based on at least one of the following:

a previous collection;

a current collection; and

an upcoming collection.

15. The program product of claim 13, wherein the program code for aggregating includes program code for constructing a comma separated value (CSV) data stream from the collected data.

16. The program product of claim 13, wherein the program code for aggregating includes program code for at least one of the following:

overwriting old date in a previous collection and inserting new data in a previous collection.

17. The program product of claim 13, wherein the program code for normalizing includes program code for at least one of the following:

compressing the aggregated data; and

converting the aggregated data to another format.

18. A method for deploying an application for managing data for data aggregation, comprising:

providing a computer infrastructure being operable to;

determine a plurality of locations of data to be collected within a source database;

acquire at least one access configuration log of the plurality of locations from which data will be collected;

simultaneously collect data from the plurality of the locations;

aggregate the collected data;

normalize the aggregated data, wherein normalizing includes adding an encryption key to the aggregated data;

store the normalized data; and

release the data at each of the plurality of locations in the source database.

19. The method of claim 18, wherein the computer infrastructure is further operable to buffer the collected data based on at least one of the following:

a previous collection;

a current collection; and

an upcoming collection.

20. The method of claim 18, wherein the computer infrastructure is further operable to perform at least one action selected from a group consisting of:

compressing the aggregated data; and

converting the aggregated data to another format.