Interest-Driven Business Intelligence Systems and Methods of Data Analysis Using Interest-Driven Data Pipelines

• US 20130238551A1
• Filed: 04/26/2013
• Published: 09/12/2013
• Est. Priority Date: 07/07/2011
• Status: Abandoned Application

• Current Assignee: Workday Incorporated
• Original Assignee: Platfora Incorporated (Workday Incorporated)
• Inventors: Eshleman, John Glenn, Werther, Benjamin Mark, Beyer, Kevin Scott, Zhang, Yewei, Baboock, Brian F.
• Application Number:
  US13/871,717
• Publication Number:
  US 20130238551A1
• Time in Patent Office: Days
• US Class Current: 707/602
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06Q 10/00 Administration; Management

Abstract

Interest-driven Business Intelligence (BI) systems in accordance with embodiments of the invention are illustrated. In one embodiment of the invention, a data processing system includes raw data storage containing raw data, metadata storage containing metadata that describes the raw data, and an interest-driven data pipeline that is automatically compiled to generate reporting data using the raw data, wherein the interest-driven data pipeline is compiled based upon reporting data requirements automatically derived from at least one report specification defined using the metadata.

Claims

1. An interest-driven business intelligence system, comprising:

raw data storage configured to contain raw data and perform extract, transform, and load (ETL) processes, where the raw data comprises unstructured data;

metadata storage configured to contain metadata that describes the raw data;

aggregate data storage configured to contain aggregate data, where aggregate data comprises structured data generated using ETL processes from the raw data;

an intermediate processing layer;

a business intelligence reporting engine;

wherein the intermediate processing layer is configured to automatically;

generate metadata describing the raw data;

derive reporting data requirements from the at least one report specification based on the metadata; and

compile an interest-driven data pipeline based upon the reporting data requirements, where compiling the interest-driven data pipeline comprises;

generating ETL processing jobs to generate aggregate data from the raw data by;

filtering the raw data using the structure of the raw data; and

applying transformations to the raw data based on the structure of the raw data;

storing the aggregate data in the aggregate data storage;

generating reporting data including reporting data satisfying the reporting data requirements using the aggregate data; and

storing the reporting data in a data mart within the intermediate processing layer for exploration by the business intelligence reporting engine;

wherein the business intelligence reporting engine is configured to;

receive metadata describing the raw data from the intermediate processing layer; and

generate a user interface enabling user exploration of the metadata to define at least one report specification, where the user exploration involves selection of additional reporting data using the metadata.

2. The interest-driven business intelligence system of claim 1, wherein the raw data storage is a data warehouse.

3. The interest-driven business intelligence system of claim 2, wherein the data warehouse is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database.

4. The interest-driven business intelligence system of claim 2, wherein the data warehouse is a distributed computing system implemented utilizing Hadoop.

5. The interest-driven business intelligence system of claim 2, wherein the data warehouse is configured to store data generated utilizing the intermediate processing layer.

6. The interest-driven business intelligence system of claim 2, wherein the intermediate processing layer is configured to generate data warehouse requests.

7. The interest-driven business intelligence system of claim 6, wherein the data warehouse requests are requests selected from the group consisting of Hive queries and MapReduce operations.

8. The interest-driven business intelligence system of claim 1, wherein the intermediate processing layer is implemented utilizing a system selected from the group consisting of a distributed computing system, a database management system, and a NoSQL database system.

9. The interest-driven business intelligence system of claim 1, wherein the business intelligence reporting engine is configured to display an indication based upon the interactive exploration of the at least one report.

10. The interest-driven business intelligence system of claim 9, wherein the indication is an estimate of the time needed to update the reporting data requirements.

11. A method for creating a report utilizing an interest-driven business intelligence system, comprising:

storing raw data in raw data storage using an interest-driven business intelligence system, where the raw data comprises unstructured data;

generating metadata describing the raw data using the interest-driven business intelligence system;

receiving report definitions generated utilizing the metadata describing the raw data using the interest-driven business intelligence system;

automatically generating reporting data requirements derived from report definitions using the interest-driven business intelligence system;

automatically compiling an interest-driven data pipeline that is part of the interest-driven business intelligence system using the reporting data requirements and the raw data;

generating reporting data from the reporting data requirements using the interest-driven data pipeline, where the reporting data comprises structured data generated using the raw data; and

storing the generated reporting data in a data mart within the interest-driven business intelligence system.