Systems and Methods for Interest-Driven Business Intelligence Systems Including Event-Oriented Data

• US 20150081618A1
• Filed: 03/05/2014
• Published: 03/19/2015
• Est. Priority Date: 09/19/2013
• Status: Active Grant

• Current Assignee: Workday Incorporated
• Original Assignee: Platfora Incorporated (Workday Incorporated)
• Inventors: Schuster, John, Werther, Benjamin Mark, Babcock, Brian, Beyer, Kevin Scott, Brown, Hans-Frederick
• Granted Patent: US 9,892,178 B2
• Time in Patent Office: Days
• US Class Current: 707/602
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 16/29 Geographical information databases

Abstract

Systems and methods for interest-driven business intelligence systems including event-oriented data in accordance with embodiments of the invention are illustrated. In one embodiment, an interest-driven business intelligence system includes raw data storage configured to contain raw data and perform ETL processes, a data mart configured to contain metadata that describes the raw data, and an intermediate processing layer, wherein the intermediate processing layer is configured to compile an interest-driven data pipeline configured to generate ETL processing jobs to generate event-oriented data from the raw data by filtering the raw data based on the metadata describing the raw data, determining ordering data based on the metadata describing the raw data, aligning the filtered raw data based on the ordering data, and generating event-oriented data based on the aligned filtered raw data, and storing the event-oriented data in the data mart.

Claims

1. An interest-driven business intelligence system, comprising:

raw data storage configured to contain raw data and perform extract, transform, and load (ETL) processes;

a data mart configured to contain metadata that describes the raw data; and

an intermediate processing layer;

wherein the intermediate processing layer is configured to automatically;

generate metadata describing the raw data;

derive reporting data requirements from at least one report specification based on the metadata; and

compile an interest-driven data pipeline based upon the reporting data requirements, where compiling the interest-driven data pipeline comprises;

generating ETL processing jobs to generate event-oriented data from the raw data by;

filtering the raw data based on the metadata describing the raw data;

determining ordering data based on the metadata describing the raw data;

aligning the filtered raw data based on the ordering data;

generating event-oriented data based on the aligned filtered raw data; and

storing the event-oriented data in the data mart;

generating reporting data including data satisfying the reporting data requirements based on the event-oriented data; and

storing the reporting data in the data mart for exploration by an interest-driven data visualization system.

2. The interest-driven business intelligence system of claim 1, wherein the raw data comprises unstructured data.

3. The interest-driven business intelligence system of claim 1, wherein the raw data storage is a data warehouse.

4. The interest-driven business intelligence system of claim 3, wherein the data warehouse is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database.

5. The interest-driven business intelligence system of claim 3, wherein the data warehouse is a distributed computing system implemented utilizing Hadoop.

6. The interest-driven business intelligence system of claim 3, wherein the data warehouse is configured to store data generated utilizing the intermediate processing layer.

7. The interest-driven business intelligence system of claim 3, wherein the intermediate processing layer is configured to generate data warehouse requests.

8. The interest-driven business intelligence system of claim 7, wherein the data warehouse requests comprise MapReduce operations.

9. The interest-driven business intelligence system of claim 1, wherein the intermediate processing layer is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database system.

10. The interest-driven business intelligence system of claim 1, further comprising an interest-driven data visualization system;

wherein the interest-driven data visualization system is configured to;

receive metadata describing the raw data from the intermediate processing layer; and

generate a user interface enabling user exploration of the metadata to define at least one report specification, where the user exploration involves selection of additional reporting data based on the metadata.

11. The interest-driven business intelligence system of claim 10, wherein the interest-driven data visualization system is configured to display an indication based upon the interactive exploration of the at least one report.

12. The interest-driven business intelligence system of claim 1, wherein the data mart is further configured to contain aggregate data, where aggregate data comprises structured data generated using ETL processes from the raw data.

13. The interest-driven business intelligence system of claim 12, wherein compiling the interest-driven pipeline further comprises generating ETL processing jobs to generate aggregate data from the raw data by:

filtering the raw data based on the metadata describing the raw data;

applying transformations to the raw data based on the metadata describing the raw data;

generating aggregate data based on the transformed data; and

storing the aggregate data in the data mart.

14. The interest-driven business intelligence system of claim 13, wherein compiling the interest-driven pipeline further comprises generating ETL processing jobs to generate aggregate data from event-oriented data by:

identifying at least one dimension within a piece of event-oriented data;

obtaining raw data corresponding to the identified at least one dimension;

applying transformations to the obtained raw data based on the metadata describing the obtained raw data;

generating aggregate data based on the transformed data; and

storing the aggregate data in the data mart.

15. The interest-driven business intelligence system of claim 13, wherein compiling the interest-driven pipeline further comprises generating ETL processing jobs to generate event-oriented data from aggregate data by:

identifying at least one dimension within a piece of aggregate data;

obtaining raw data corresponding to the identified at least one dimension;

filtering the obtained raw data based on the metadata describing the obtained raw data;

determining ordering data based on the metadata describing the obtained raw data;

aligning the filtered obtained raw data based on the ordering data;

generating event-oriented data based on the aligned data; and

storing the event-oriented data in the data mart.

16. The interest-driven business intelligence system of claim 1, wherein the data mart is contained within the intermediate processing layer.

17. The interest-driven business intelligence system of claim 1, wherein the ordering data comprises at least one dimension identified in the metadata describing the raw data.

18. The interest-driven business intelligence system of claim 17, wherein aligning the filtered raw data based on the ordering data comprises sorting the event-oriented data according to the facts associated with the ordering data in each piece of event-oriented data.

19. The interest-driven business intelligence system of claim 18, wherein:

aligning the filtered raw data based on the ordering data further comprises mapping at least two dimensions within the filtered raw data to a common set of data; and

the common set of data is based on the ordering data.

20. The interest-driven business intelligence system of claim 1, wherein the ordering data comprises time data.

21. A method for creating a report utilizing an interest-driven business intelligence system, comprising:

storing raw data in raw data storage using an interest-driven business intelligence system, where the raw data comprises unstructured data;

generating metadata describing the raw data using the interest-driven business intelligence system;

receiving report definitions generated utilizing the metadata describing the raw data using the interest-driven business intelligence system;

automatically generating reporting data requirements derived from report definitions using the interest-driven business intelligence system;

automatically compiling an interest-driven data pipeline that is part of the interest-driven business intelligence system using the reporting data requirements and the raw data;

obtaining event-oriented data from the interest-driven data pipeline based on the generated reporting data requirements using the interest-driven business intelligence system, where the event-oriented data comprises a set of dimensions and the event-oriented data is ordered based on at least one dimension in the set of dimensions;

generating reporting data from the reporting data requirements using the interest-driven data pipeline, where the reporting data is based on a portion of the event-oriented data; and

storing the generated reporting data in a data mart within the interest-driven business intelligence system.

22. The method of claim 21, wherein compiling the interest-driven pipeline further comprises generating ETL processing jobs to generate aggregate data from event-oriented data by:

identifying at least one dimension within a piece of event-oriented data using the interest-driven business intelligence system;

obtaining raw data corresponding to the identified at least one dimension using the interest-driven business intelligence system;

applying transformations to the obtained raw data based on the metadata describing the obtained raw data to generate aggregate data using the interest-driven business intelligence system; and

storing the aggregate data using the interest-driven business intelligence system.

23. The method of claim 22, wherein generating ETL processing jobs to generate aggregate data from the raw data further comprises:

filtering the raw data based on the metadata describing the raw data using the interest-driven business intelligence system;

applying transformations to the raw data based on the metadata describing the raw data to generate aggregate data using the interest-driven business intelligence system; and

storing the aggregate data using the interest-driven business intelligence system.

24. The method of claim 22, wherein compiling the interest-driven pipeline further comprises generating ETL processing jobs to generate event-oriented data from aggregate data by:

identifying at least one dimension within a piece of aggregate data using the interest-driven business intelligence system;

obtaining raw data corresponding to the identified at least one dimension using the interest-driven business intelligence system;

filtering the obtained raw data based on the metadata describing the obtained raw data using the interest-driven business intelligence system;

determining ordering data based on the metadata describing the obtained raw data using the interest-driven business intelligence system;

aligning the filtered obtained raw data based on the ordering data to generate event-oriented data using the interest-driven business intelligence system; and

storing the event-oriented data using the interest-driven business intelligence system.

25. The method of claim 24, wherein the ordering data comprises time data.

26. An interest-driven business intelligence system, comprising:

raw data storage configured to contain raw data and perform extract, transform, and load (ETL) processes, where the raw data comprises unstructured data;

an intermediate processing layer;

a data mart configured to contain metadata that describes the raw data and contained within the intermediate processing layer; and

an interest-driven data visualization system;

wherein the intermediate processing layer is configured to automatically;

generate metadata describing the raw data;

derive reporting data requirements from at least one report specification based on the metadata; and

compile an interest-driven data pipeline based upon the reporting data requirements, where compiling the interest-driven data pipeline comprises;

generating ETL processing jobs to generate event-oriented data from the raw data by;

filtering the raw data based on the metadata describing the raw data;

determining ordering data based on the metadata describing the raw data, where the ordering data comprises time data;

aligning the filtered raw data based on the ordering data;

generating event-oriented data based on the aligned filtered raw data; and

storing the event-oriented data in the data mart;

generating ETL processing jobs to generate aggregate data from the raw data by;

filtering the raw data based on the metadata describing the raw data;

applying transformations to the raw data based on the metadata describing the raw data;

generating aggregate data based on the transformed data; and

storing the aggregate data in the data mart;

generating ETL processing jobs to generate aggregate data from event-oriented data by;

identifying at least one dimension within a piece of event-oriented data;

obtaining raw data corresponding to the identified at least one dimension;

applying transformations to the obtained raw data based on the metadata describing the obtained raw data;

generating aggregate data based on the transformed data; and

storing the aggregate data in the data mart;

generating ETL processing jobs to generate event-oriented data from aggregate data by;

identifying at least one dimension within a piece of aggregate data;

obtaining raw data corresponding to the identified at least one dimension;

filtering the obtained raw data based on the metadata describing the obtained raw data;

determining ordering data based on the metadata describing the obtained raw data;

aligning the filtered obtained raw data based on the ordering data;

generating event-oriented data based on the aligned data; and

storing the event-oriented data in the data mart;

generating reporting data including data satisfying the reporting data requirements based on the event-oriented data;

generating reporting data including data satisfying the reporting data requirements based on the aggregate data; and

storing the reporting data in the data mart for exploration by an interest-driven data visualization system;

wherein the raw data storage is a data warehouse; and

wherein the interest-driven data visualization system is configured to;

receive metadata describing the raw data from the intermediate processing layer; and

generate a user interface enabling user exploration of the metadata to define at least one report specification, where the user exploration involves selection of additional reporting data based on the metadata.