Systems and methods for interest-driven data sharing in interest-driven business intelligence systems

• US 9,767,173 B2
• Filed: 03/08/2013
• Issued: 09/19/2017
• Est. Priority Date: 10/22/2012
• Status: Active Grant

• Current Assignee: Workday Incorporated
• Original Assignee: Workday Incorporated
• Inventors: Eshleman, John Glenn, Beyer, Kevin Scott, Babcock, Brian, Kanne, Carl-Christian
• Primary Examiner(s): Mobin, Hasanul
• Application Number:
  US13/791,042
• Publication Number:
  US 20140114971A1
• Time in Patent Office: 1,656 Days
• Field of Search: 707736, 707738, 707776
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Systems and methods for interest-driven data sharing in interest-driven business intelligence systems in accordance with embodiments of the invention are illustrated. In one embodiment, an interest-driven data sharing server system includes a processor, a memory configured to store an interest-driven data sharing application, raw data storage, metadata storage configured to store data description metadata describing the raw data, and report storage configured to store previously generated reports, wherein the interest-driven data sharing application configures the processor to receive a report specification, locate at least one previously generated report stored in the report storage, identify raw data stored in the raw data storage using the data description metadata and at least one of the reporting data requirements, calculate redundant data metadata using the located previously generated reports and the identified raw data, determine modified reporting data requirements, retrieve updated source data, and generate reporting data based on the updated source data.

Claims

1. An interest-driven data sharing server system, comprising:

raw data storage storing raw data, where a portion of the raw data comprises unstructured data;

an interest-driven data sharing server comprising;

a processor;

a memory connected to the processor and storing an interest-driven data sharing application;

aggregate data storage storing;

data description metadata describing the raw data; and

previously generated reports comprising previous reporting data and previous reporting data requirements;

wherein the interest-driven data sharing application directs the processor to;

receive a report specification, where the report specification comprises at least one reporting data requirement;

locate at least one previously generated report stored in the report aggregate data storage based upon the at least one data requirement of the report specification and the previous reporting data requirements of the reports stored in the aggregate data storage;

identify raw data stored in the raw data storage that satisfies the at least one data reporting requirement of the report specification using the at least one reporting data requirement and the data description metadata describing the raw data stored in the aggregate data storage;

calculate redundant data metadata between the located at least one previously generated report and the identified raw data, wherein the redundant data metadata identifies data common to the identified raw data and the previous reporting data in the located at least one previously generated report, wherein the at least one reporting data requirement comprises at least one reporting data dimension where each of the at least one reporting data dimensions corresponds to one of the plurality of dimensions associated with the previously generated reports, and wherein the calculating of the redundant data metadata between the located at least one previously generated report and the identified raw data comprises to;

build a data dimension dictionary comprising a plurality of data dimensions associated with the previously generated reports; and

compare the data dimension directory and the at least one reporting data requirement;

determine modified reporting data requirements using the redundant data metadata, where the modified reporting data requirements exclude the common data identified by the redundant data metadata;

retrieve updated source data from the raw data storage using the modified reporting data requirements; and

generate reporting data for the report specification from the updated source data and the previous reporting data contained in the at least one previously generated report using the at least one data requirement of the report specification.

2. The system of claim 1, wherein the interest-driven data sharing application directs the processor to generate the reporting data utilizing an interest-driven data pipeline, where the interest-driven data pipeline is configured to aggregates the updated source data and the previous reporting data of the located at least one previously generated report based on the report specification.

3. The system of claim 1, wherein:

the raw data storage is separate and remote from the interest-driven data sharing server; and

the interest-driven data sharing application directs the processor to generate a data source job that directs the raw data storage to retrieve the updated source data from the raw data storage.

4. The system of claim 3, wherein the data source job directs the raw data storage to generate aggregate data based on raw data, where a portion of the raw data comprises unstructured data.

5. The system of claim 1, wherein the interest-driven data sharing application directs the processor to update the data dimension directory based on the generated reporting data.

6. The system of claim 1, wherein the interest-driven data sharing application directs the processor to:

retrieve the updated source data from the raw data storage using the modified reporting data requirements, where the updated source data fulfills a portion of the modified reporting data requirement;

generate partial reporting data from the updated source data and the previous reporting data contained in at least one of the at least one previously generated reports based on the report specification;

retrieve supplemental source data from the raw data storage, where the supplemental source data fulfills a remaining portion of the modified reporting data requirement; and

generate new reporting data from the partial reporting data and the supplemental source data based on the report specification.

7. The system of claim 6, wherein:

the aggregate data storage stores aggregate data;

the updated source data comprises aggregate data; and

the interest-driven data sharing application directs the processor to store the updated source data using the aggregate data storage.

8. The system of claim 1, wherein:

at least one of the at least one reporting data requirement of the report specification comprises data source metadata identifying at least one data source, where the at least one data source stores raw data; and

the interest-driven data sharing application directs the processor to retrieve updated source data from the at least one data source.

9. The system of claim 1, wherein:

the report data storage is separate from the interest-driven data sharing server; and

the interest-driven data sharing application directs the processor to;

generate reporting data based on the updated source data and the previous reporting data contained in at least one of the at least one previously generated reports based on the data mapping metadata.

10. A method for interest-driven data sharing, comprising:

storing at least one previously generated report where each at least one previously generated report includes previous reporting data and at least one previous reporting data requirement using an aggregate storage;

storing data description metadata describing raw data stored using a raw data storage using the aggregate storage;

receiving a report specification using an interest-driven data sharing server system, where the report specification comprises at least one reporting data requirement;

locating at least one previously generated report based on the at least one previously reporting data requirement of the at least one previously generated report and the at least one reporting data requirement of the report specification using the interest-driven data sharing server system;

identifying raw data stored in the raw data storage based on the data description metadata and at least one of the reporting data requirements using the interest-driven data sharing server system;

calculating redundant data metadata based on the at least one located previously generated reports and the identified raw data using the interest-driven data sharing server system, where the redundant data metadata identifies data common to the identified raw data and the previous reporting data in the at least one located previously generated report, wherein the at least one reporting data requirement comprises at least one reporting data dimension where each of the at least one reporting data dimensions corresponds to one of the plurality of dimensions associated with the previously generated reports, and wherein the calculating of the redundant data metadata between the located at least one previously generated report and the identified raw data comprises;

building a data dimension dictionary comprising a plurality of data dimensions associated with the previously generated reports; and

comparing the data dimension directory and the at least one reporting data requirement;

determining modified reporting data requirements based on the redundant data metadata using the interest-driven data sharing server system, where the modified reporting data requirements exclude the common data identified by the redundant data metadata;

retrieving updated source data from raw data storage based on the modified reporting data requirements using the interest-driven data sharing server system; and

generating reporting data for the report specification based on the updated source data and reporting data contained in the at least one located previously generated reports using the interest-driven data sharing server system.

11. The method of claim 10, further comprising generating the reporting data utilizing an interest-driven data pipeline within the interest-driven data sharing server system, where the interest-driven data pipeline aggregates the updated source data and the previous reporting data based on the at least one reporting data requirement of the report specification.

12. The method of claim 10, further comprising using the interest-driven data pipeline to generate a data source job configured to retrieve the updated source data from the raw data storage system that is separate and remote from the interest-driven data sharing server system using the interest-driven data sharing server system.

13. The method of claim 12, wherein the data source job directs the raw data storage to generate aggregate data based on raw data, where a portion of the raw data comprises unstructured data.

14. The method of claim 10, further comprising updating the data dimension directory based on the generated reporting data using the interest-driven data sharing server system.

15. The method of claim 10, further comprising:

retrieving the updated source data from the raw data storage based on the modified reporting data requirements using the interest-driven data sharing server system, where the updated source data fulfills a portion of the modified reporting data requirement;

generating partial reporting data based on the updated source data and the previous reporting data contained in the at least one located previously generated reports using the interest-driven data sharing server system;

retrieving supplemental source data from the raw data storage using the interest-driven data sharing server system, where the supplemental source data fulfills the remaining portion of the modified reporting data requirement; and

generating the reporting data based on the partial reporting data and the supplemental source data using the interest-driven data sharing server system.

16. The method of claim 15, further comprising storing the updated source data using the interest-driven data sharing server system.

17. The method of claim 10, further comprising retrieving the updated source data from at least one data source using the interest-driven data sharing server system, where at least one of the at least one reporting data requirement of the report specification comprises data source metadata identifying the at least one data source storing raw data.

18. The method of claim 10, further comprising:

generating reporting data based on the updated source data, the previous reporting data contained in at least one of the at least one previously generated reports, and the data mapping metadata using the interest-driven data sharing server system.