Systems and Methods for Data Ingest in Interest-Driven Business Intelligence Systems

• US 20160162521A1
• Filed: 07/14/2015
• Published: 06/09/2016
• Est. Priority Date: 12/08/2014
• Status: Active Application

• Current Assignee: Workday Incorporated
• Original Assignee: Platfora Incorporated (Workday Incorporated)
• Inventors: Lin, Honray, Pradhan, Mayank, Beyer, Kevin Scott, Brown, Hans-Frederick
• Application Number:
  US14/799,373
• Publication Number:
  US 20160162521A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Systems and methods for data ingest in interest-driven business intelligence systems in accordance with embodiments of the invention are illustrated. The interest-driven business intelligence system may maintain a set of registered data ingest instruction data that includes at least one registered data ingest instruction data. Each of the at least one registered data ingest instruction data includes an identifier and data ingest instruction data associated with the identifier. The system may receive a request to generate data using registered data instruction data. The request may include the identifier of the registered data instruction data. Data is generated using the data ingest instruction data associated with the requested identifier and at least one of raw data, source data, and aggregate data, and provided for use.

Claims

1. An interest-driven business intelligence server system comprising:

a processor;

memory connected to the processor that stores an interest-driven business intelligence application, source data, aggregate data, and data ingest instruction data;

data storage that stores raw data; and

wherein the interest-driven business intelligence application directs the processor to;

maintain a set of registered data ingest instruction data that includes at least one registered data ingest instruction data wherein each of the at least one registered data ingest instruction data includes an identifier and data ingest instruction data associated with the identifier;

receive a request to generate data using registered data instruction data wherein request includes the identifier of the registered data instruction data;

generate data using the data ingest instruction data associated with the requested identifier and at least one of raw data, source data, and aggregate data, and

provide the generated data for use.

2. The interest-driven business intelligence server system of claim 1, wherein the interest-driven business intelligence application further directs the processor to:

analyze the generated data;

generate statistic data for the generated statistics for the generated data; and

provide the statistic data for use.

3. The interest-driven business intelligence server system of claim 2, wherein the statistic data is provided as metadata associated with the generated data.

4. The interest-driven business intelligence server system of claim 1, wherein the generating of the data using data ingest instruction data comprises updating a set of data generated using the data ingest instruction data associated with the identifier.

5. The interest-driven business intelligence server system of claim 1, wherein the interest-driven business intelligence application further directs the processor to store the generated data in memory.

6. The interest-driven business intelligence server system of claim 1, wherein the interest-driven business intelligence application further directs the processor to:

receive a request to register data ingest instruction data;

receive an identifier associated with the data ingest instruction data to register;

receive code written in a supported language to generate the data ingest instruction data;

compile the code to generate the data ingest instruction data; and

store the data ingest instruction data and the associated identifier as registered data ingest instruction data in memory.

7. The interest-driven business intelligence server system of claim 6, wherein the interest-driven business intelligence application further directs the processor to:

generate data using the data ingest instruction data associated with the identifier in response to compiling the code to generate the data ingest instruction data; and

store the generated data in memory as part of a data catalog maintained in memory, wherein the data is associated with the identifier in the data catalog.

8. The interest-driven business intelligence server system of claim 1, wherein the registered data ingest instruction data is a function to perform on a set of data.

9. The interest-driven business intelligence server system of claim 8, wherein the interest-driven business intelligence application further directs the processor to:

receive an identification of a set of data to which the registered data ingest instruction data is to be applied;

obtain the set of data;

apply the ingest instruction data associated with the identifier to the set of data to generate data.

10. The interest-driven business intelligence server system of claim 9, wherein:

the interest-driven business intelligence application further directs the processor to receive a change to at least one variable in a set of parameters for the data ingest instruction data exposed for use; and

the ingest instruction data is applied with to the data set using the change to the at least one variable in the set of parameter exposed for use.

11. The interest-driven business intelligence server system of claim 8, wherein the interest-driven business intelligence application further directs the processor to:

receive a request to register data ingest instruction data that provides a function;

receive an identifier associated with the data ingest instruction data to register;

receive code written in a supported language to generate the data ingest instruction data;

receive a set of parameters including at least one variable for the data ingest instruction data that provides the function to expose to a user to allow the user to change;

compile the code to generate the data ingest instruction data; and

store the data ingest instruction data, the exposed set of parameters and the associated identifier as registered data ingest instruction data in memory.

12. A method performed by an interest-driven business intelligence server system to provide data for an interest-driven data pipeline comprising:

maintaining a set of registered data ingest instruction data using the interest-driven business intelligence server system that includes at least one registered data ingest instruction data wherein each of the at least one registered data ingest instruction data includes an identifier and data ingest instruction data associated with the identifier;

receiving a request using the interest-driven business intelligence server system to generate data using registered data instruction data wherein request includes the identifier of the registered data instruction data; and

generating data using the interest-driven business intelligence server system from the data ingest instruction data associated with the requested identifier and at least one of raw data, source data, and aggregate data, and

providing the generated data to the interest-driven data pipeline using the interest-driven business intelligence server system.

13. The method of claim 12, further comprising:

analyzing the generated data using the interest-driven business intelligence server system;

generating statistic data for the generated statistics for the generated data using the interest-driven business intelligence server system; and

providing the statistic data to the interest-driven data pipeline using the interest-driven business intelligence server system.

14. The method of claim 13, wherein the statistic data is provided as metadata associated with the generated data.

15. The method of claim 12, wherein the generating of the data from the data ingest instruction data comprises updating a set of data generated using the interest-driven business intelligence server system based upon the data ingest instruction data associated with the identifier.

16. The method of claim 12, further comprising storing the generated data in memory.

17. The method of claim 12, further comprising:

receiving a request to register data ingest instruction data using the interest-driven business intelligence server system;

receiving an identifier associated with the data ingest instruction data to register using the interest-driven business intelligence server system;

receiving code written in a supported language to generate the data ingest instruction data using the interest-driven business intelligence server system;

compiling the code to generate the data ingest instruction data using the interest-driven business intelligence server system; and

storing the data ingest instruction data and the associated identifier as registered data ingest instruction data in memory using the interest-driven business intelligence server system.

18. The method of claim 17, further comprising:

generating data using the data ingest instruction data associated with the identifier in response to compiling the code to generate the data ingest instruction data using the interest-driven business intelligence server system; and

storing the generated data in memory as part of a data catalog maintained in memory, wherein the data is associated with the identifier in the data catalog using the interest-driven business intelligence server system.

19. The method of claim 12, wherein the registered data ingest instruction data is a function to perform on a set of data.

20. The method of claim 19, further comprising:

receiving an identification of a set of data to which the registered data ingest instruction data is to be applied in the interest-driven business intelligence server system;

obtaining the set of data using the interest-driven business intelligence server system;

applying the ingest instruction data associated with the identifier to the set of data to generate data using the interest-driven business intelligence server system.

21. The method of claim 20, further comprising:

receiving a change to at least one variable in a set of parameters for the data ingest instruction data exposed for use using the interest-driven business intelligence server system; and

wherein the ingest instruction data is applied with to the data set using the change to the at least one variable in the set of parameter exposed for use in the interest-driven business intelligence server system.

22. The method of claim 19, further comprising:

receiving a request to register data ingest instruction data that provides a function using the interest-driven business intelligence server system;

receiving an identifier associated with the data ingest instruction data to register in the interest-driven business intelligence server system;

receiving code written in a supported language to generate the data ingest instruction data using the interest-driven business intelligence server system;

receiving a set of parameters including at least one variable for the data ingest instruction data that provides the function to expose to a user to allow the user to change using the interest-driven business intelligence server system;

compiling the code to generate the data ingest instruction data using the interest-driven business intelligence server system; and

storing the data ingest instruction data, the exposed set of parameters and the associated identifier as registered data ingest instruction data in memory using the interest-driven business intelligence server system.