Systems and Methods for Interest-Driven Data Sharing in Interest-Driven Business Intelligence Systems

• US 20140114971A1
• Filed: 03/08/2013
• Published: 04/24/2014
• Est. Priority Date: 10/22/2012
• Status: Active Grant

• Current Assignee: Workday Incorporated
• Original Assignee: Platfora Incorporated (Workday Incorporated)
• Inventors: Eshleman, John Glenn, Beyer, Kevin Scott, Babcock, Brian, Kanne, Carl-Christan
• Granted Patent: US 9,767,173 B2
• Time in Patent Office: Days
• US Class Current: 707/736
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Systems and methods for interest-driven data sharing in interest-driven business intelligence systems in accordance with embodiments of the invention are illustrated. In one embodiment, an interest-driven data sharing server system includes a processor, a memory configured to store an interest-driven data sharing application, raw data storage, metadata storage configured to store data description metadata describing the raw data, and report storage configured to store previously generated reports, wherein the interest-driven data sharing application configures the processor to receive a report specification, locate at least one previously generated report stored in the report storage, identify raw data stored in the raw data storage using the data description metadata and at least one of the reporting data requirements, calculate redundant data metadata using the located previously generated reports and the identified raw data, determine modified reporting data requirements, retrieve updated source data, and generate reporting data based on the updated source data.

Claims

1. An interest-driven data sharing server system, comprising:

a processor;

a memory connected to the processor and configured to store an interest-driven data sharing application;

raw data storage configured to store raw data, where a portion of the raw data comprises unstructured data;

metadata storage configured to store data description metadata describing the raw data; and

report storage configured to store previously generated reports comprising previous reporting data and previous reporting data requirements;

wherein the interest-driven data sharing application configures the processor to;

receive a report specification, where the report specification comprises at least one reporting data requirement;

locate at least one previously generated report stored in the report storage using the previously reporting data requirements and the at least one reporting data requirement;

identify raw data stored in the raw data storage using the data description metadata and at least one of the reporting data requirements;

calculate redundant data metadata using the located previously generated reports and the identified raw data, where the redundant data metadata identifies data common to the identified raw data and the previous reporting data in the located previously generated reports;

determine modified reporting data requirements, where the modified reporting data requirements exclude the common data identified by the redundant data metadata;

retrieve updated source data from the raw data storage using the modified reporting data requirements; and

generate reporting data based on the updated source data and the previous reporting data contained in at least one of the at least one previously generated reports.

2. The system of claim 1, wherein the interest-driven data sharing application configures the processor to generate the reporting data utilizing an interest-driven data pipeline, where the interest-driven data pipeline is configured to aggregate the updated data and the previous reporting data based on the report specification.

3. The system of claim 1, wherein:

the raw data storage is separate and remote from the interest-driven data sharing server system; and

the interest-driven data sharing application configures the processor to generate a data source job configured to retrieve updated source data from the raw data storage.

4. The system of claim 3, wherein the data source job configures the raw data storage to generate aggregate data based on raw data, where a portion of the raw data comprises unstructured data.

5. The system of claim 1, further comprising a data dimension directory, where the data dimension directory comprises a plurality of data dimensions associated with the previously generated reports;

wherein the at least one reporting data requirement comprises at least one reporting data dimension; and

wherein the interest-driven data sharing application configures the processor to calculate redundant metadata based on the data dimension directory and the at least one reporting data requirement.

6. The system of claim 5, wherein the interest-driven data sharing application configures the processor to update the data dimension directory based on the generated reporting data.

7. The system of claim 1, wherein the interest-driven data sharing application configures the processor to:

retrieve updated source data from the raw data storage using the modified reporting data requirements, where the updated source data fulfills a portion of the modified reporting data requirement;

generate partial reporting data based on the updated source data and the previous reporting data contained in at least one of the at least one previously generated reports;

retrieve supplemental source data from the raw data storage, where the supplemental source data fulfills the remaining portion of the modified reporting data requirement; and

generate reporting data using the partial reporting data and the supplemental source data.

8. The system of claim 7, further comprising aggregate data storage configured to store aggregate data;

wherein the updated source data comprises aggregate data; and

wherein the interest-driven data sharing application configures the processor to store the updated source data using the aggregate data storage.

9. The system of claim 1, wherein:

at least one of the at least one reporting data requirement comprises data source metadata identifying at least one data source, where the at least one data source is configured to store raw data; and

the interest-driven data sharing application configures the processor to retrieve updated source data from the at least one data source.

10. The system of claim 1, wherein:

the report data storage is separate from the interest-driven data sharing server system; and

the interest-driven data sharing application configures the processor to;

generate data mapping metadata, where the data mapping metadata identifies correspondences between the previous reporting data requirements and the received report specification using the data description metadata;

calculate redundant data metadata using the located previously generated reports and the identified raw data based on the data mapping metadata; and

generate reporting data based on the updated source data and the previous reporting data contained in at least one of the at least one previously generated reports based on the data mapping metadata.

11. A method for interest-driven data sharing, comprising:

receiving a report specification using an interest-driven data sharing server system, where the report specification comprises at least one reporting data requirement;

locating at least one previously generated report stored in the report storage based on the previously reporting data requirements and the at least one reporting data requirement using the interest-driven data sharing server system;

identifying raw data stored in the raw data storage based on the data description metadata and at least one of the reporting data requirements using the interest-driven data sharing server system;

calculating redundant data metadata based on the located previously generated reports and the identified raw data using the interest-driven data sharing server system, where the redundant data metadata identifies data common to the identified raw data and the previous reporting data in the located previously generated reports;

determining modified reporting data requirements using the interest-driven data sharing server system, where the modified reporting data requirements exclude the common data identified by the redundant data metadata;

retrieving updated source data based on the modified reporting data requirements using the interest-driven data sharing server system; and

generating reporting data based on the updated source data and the previous reporting data contained in at least one of the at least one previously generated reports using the interest-driven data sharing server system.

12. The method of claim 11, further comprising generating the reporting data utilizing an interest-driven data pipeline within the interest-driven data sharing server system, where the interest-driven data pipeline is configured to aggregate the updated data and the previous reporting data based on the report specification.

13. The method of claim 11, further comprising configuring the interest-driven data pipeline to generate a data source job configured to retrieve updated source data from a raw data storage system separate and remote from the interest-driven data sharing server system using the interest-driven data sharing server system.

14. The method of claim 13, wherein the data source job configures the raw data storage to generate aggregate data based on raw data, where a portion of the raw data comprises unstructured data.

15. The method of claim 11, further comprising calculating redundant metadata based on the data dimension directory and the at least one reporting data requirement using the interest-driven data sharing server system, where the data dimension directory, where the data dimension directory comprises a plurality of data dimensions associated with the previously generated reports and the at least one reporting data requirement comprises at least one reporting data dimension.

16. The method of claim 15, further comprising updating the data dimension directory based on the generated reporting data using the interest-driven data sharing server system.

17. The method of claim 11, further comprising:

retrieving updated source data based on the modified reporting data requirements using the interest-driven data sharing server system, where the updated source data fulfills a portion of the modified reporting data requirement;

generating partial reporting data based on the updated source data and the previous reporting data contained in at least one of the at least one previously generated reports using the interest-driven data sharing server system;

retrieving supplemental source data using the interest-driven data sharing server system, where the supplemental source data fulfills the remaining portion of the modified reporting data requirement; and

generating reporting data based on the partial reporting data and the supplemental source data using the interest-driven data sharing server system.

18. The method of claim 17, further comprising storing the updated source data using the interest-driven data sharing server system.

19. The method of claim 11, further comprising retrieving updated source data from at least one data source using the interest-driven data sharing server system, where at least one of the at least one reporting data requirement comprises data source metadata identifying the at least one data source configured to store raw data.

20. The method of claim 11, further comprising:

generating data mapping metadata using the interest-driven data sharing server system, where the data mapping metadata identifies correspondences between the previous reporting data requirements and the received report specification using the data description metadata;

calculating redundant data metadata based on the located previously generated reports, the identified raw data, and the data mapping metadata using the interest-driven data sharing server system; and

generating reporting data based on the updated source data, the previous reporting data contained in at least one of the at least one previously generated reports, and the data mapping metadata using the interest-driven data sharing server system.