Interest-driven business intelligence systems and methods of data analysis using interest-driven data pipelines
First Claim
Patent Images
1. An interest-driven business intelligence system, comprising:
- a raw data storage system configured to contain raw data and perform extract, transform, and load (ETL) processes, where the raw data comprises unstructured data;
metadata storage configured to contain metadata that describes the raw data;
aggregate data storage configured to contain aggregate data, where aggregate data comprises structured data generated using ETL processes from the raw data;
an intermediate processing layer;
a business intelligence reporting engine;
wherein the intermediate processing layer is configured to automatically;
generate metadata describing the raw data;
derive reporting data requirements from the at least one report specification based on the metadata; and
automatically compile an interest-driven data pipeline based upon the reporting data requirements, where automatically compiling the interest-driven data pipeline comprises;
generating ETL processing jobs to generate aggregate data from the raw data by;
filtering the raw data using the structure of the raw data; and
applying transformations to the raw data based on the structure of the raw data;
storing the aggregate data in the aggregate data storage;
generating reporting data including reporting data satisfying the reporting data requirements using the aggregate data; and
storing the reporting data in a data mart within the intermediate processing layer for exploration by the business intelligence reporting engine;
wherein the business intelligence reporting engine is configured to;
receive metadata describing the raw data from the intermediate processing layer; and
generate a user interface enabling user exploration of the metadata to define at least one report specification, where the user exploration involves selection of additional reporting data using the metadata; and
wherein the intermediate processing layer is further configured to;
automatically update the reporting data requirements based upon the additional reporting data selected using the metadata via the business intelligence reporting engine;
automatically recompile the interest-driven data pipeline in real time to generate the additional reporting data selected using the metadata in response to the changes in the updated reporting data requirements by;
automatically generating ETL jobs to generate updated aggregate data from the raw data and providing the ETL jobs to the raw data storage;
automatically generating the additional reporting data using the updated aggregate data; and
storing the additional reporting data in the data mart within the intermediate processing layer that includes the previously generated reporting; and
wherein the business intelligence reporting engine is further configured to generate at least one report based upon the at least one report specification using the reporting data stored within the data mart within the intermediate processing layer.
2 Assignments
0 Petitions
Accused Products
Abstract
Interest-driven Business Intelligence (BI) systems in accordance with embodiments of the invention are illustrated. In one embodiment of the invention, a data processing system includes raw data storage containing raw data, metadata storage containing metadata that describes the raw data, and an interest-driven data pipeline that is automatically compiled to generate reporting data using the raw data, wherein the interest-driven data pipeline is compiled based upon reporting data requirements automatically derived from at least one report specification defined using the metadata.
24 Citations
10 Claims
-
1. An interest-driven business intelligence system, comprising:
-
a raw data storage system configured to contain raw data and perform extract, transform, and load (ETL) processes, where the raw data comprises unstructured data; metadata storage configured to contain metadata that describes the raw data; aggregate data storage configured to contain aggregate data, where aggregate data comprises structured data generated using ETL processes from the raw data; an intermediate processing layer; a business intelligence reporting engine; wherein the intermediate processing layer is configured to automatically; generate metadata describing the raw data; derive reporting data requirements from the at least one report specification based on the metadata; and automatically compile an interest-driven data pipeline based upon the reporting data requirements, where automatically compiling the interest-driven data pipeline comprises; generating ETL processing jobs to generate aggregate data from the raw data by; filtering the raw data using the structure of the raw data; and applying transformations to the raw data based on the structure of the raw data; storing the aggregate data in the aggregate data storage; generating reporting data including reporting data satisfying the reporting data requirements using the aggregate data; and storing the reporting data in a data mart within the intermediate processing layer for exploration by the business intelligence reporting engine; wherein the business intelligence reporting engine is configured to; receive metadata describing the raw data from the intermediate processing layer; and generate a user interface enabling user exploration of the metadata to define at least one report specification, where the user exploration involves selection of additional reporting data using the metadata; and wherein the intermediate processing layer is further configured to; automatically update the reporting data requirements based upon the additional reporting data selected using the metadata via the business intelligence reporting engine; automatically recompile the interest-driven data pipeline in real time to generate the additional reporting data selected using the metadata in response to the changes in the updated reporting data requirements by; automatically generating ETL jobs to generate updated aggregate data from the raw data and providing the ETL jobs to the raw data storage; automatically generating the additional reporting data using the updated aggregate data; and storing the additional reporting data in the data mart within the intermediate processing layer that includes the previously generated reporting; and wherein the business intelligence reporting engine is further configured to generate at least one report based upon the at least one report specification using the reporting data stored within the data mart within the intermediate processing layer. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
Specification