Plug-in architecture of business intelligence platform using data source cartridges

• US 10,067,978 B2
• Filed: 10/07/2014
• Issued: 09/04/2018
• Est. Priority Date: 10/07/2014
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Park, Hoyong, Bolsius, Roger, Dhakar, Ashok Kumar, Velusamy, Muthukumar
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Hoang, Hau H
• Application Number:
  US14/508,148
• Publication Number:
  US 20160098449A1
• Time in Patent Office: 1,428 Days
• CPC Class Codes: G06F 16/24534 Query rewriting; Transformation
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/256 in federated or virtual databases
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

Embodiments implement a plug-in architecture supporting adding new data sources to a federated database without the need to recode a federated database manager software module. The federated database manager software module includes a plug-in framework for interfacing with a data source cartridge. The data source cartridge supplies metadata including a feature table of a newly added data source to the framework and creates a physical query for the newly added data source based on an execution plan generated by the federated database manager software module. The data source cartridge returns the result set of the query of the new added data source to the federated database manager software module.

Claims

1. A method performed by a federated data source manager functional component executed by a computer system, with the federated data source manager functional component configured to receive a logical query to a federated database, where the federated database aggregates a predetermined set of predetermined data sources into a logical model, the method comprising:

adding a new data source to the federated database, wherein the federated data source manager functional component includes separate gateways for each of the predetermined data sources;

plugging into a data source server that communicates with the federated data source, manager functional component, a new data source cartridge for the new data source;

retrieving, by the federated data source manager functional component, metadata from a data source cartridge, with the data source cartridge configured to supply metadata of an added date source associated with the data source cartridge, where the added data source is not included in the predetermined set of predetermined data sources;

generating physical execution plan, by the federated data source manager functional component and using at least a portion of the metadata retrieved from the data source cartridge, for the logical query with the physical execution plan including at least one query to the added date source;

providing, by the federated data source manager functional component, at least a part of the physical execution plan to the data source cartridge to facilitate generating a physical query, by the data source cartridge, to the added data source to implement the part of the physical execution plan, wherein the federated data source manager functional component does not include program code for generating a physical query to the added data source; and

receiving, by the federated data source manager functional component and from the data source cartridge, a result set from the physical query to the added data source, wherein the data source cartridge is used instead of a code generator, as execution engine, and one of the separate gateways of the federated data source manager functional component to provide the result set; and

using the code generator, the execution engine, and the separate gateways to provide result sets from the predetermined data sources.

2. The method of claim 1 wherein providing at least a part of the physical execution plan to the data source cartridge comprises:

transforming at least a portion of the physical execution plan to a first Extensible Markup Language (XML) representation.

3. The method of claim 2 wherein the added data source is a Java data source and further comprising:

transforming the first XML representation of the physical execution plan to a Java-side execution plan.

4. The method of claim 1 wherein retrieving metadata from the data source cartridge further comprises:

receiving a feature list of the added data source.

5. The method of claim 1 wherein retrieving metadata from the data source cartridge further comprises:

receiving metadata of a new connection pool type not supported by the federated data source manager functional component and wherein the physical execution plan includes at least one query utilizing the new connection pool type.

6. A method, performed by a plug-in framework functional component of a federated data source manager functional component executed by a computer system, wherein the federated data source manager functional component is configured to federate only a predetermined set of predetermined data sources into a federated database and includes an execution function component configured to generate physical queries to only data sources in the predetermined set and wherein the federated data source manager functional component includes an analytics component for creating a physical execution plan from a received logical query, the method comprising:

providing to the analytics component of the federated data source manager functional component, by the plug-in framework functional component, metadata of an added data source received from a data source cartridge associated with the added data source, where the added data source is not included in the predetermined set of predetermined data sources and where the analytics component of the federated data source manager functional component is configured to utilize the metadata to generate the physical execution plan including the added data source, wherein the federated data source manager functional component includes separate gateways for each of the predetermined data sources and wherein the data source cartridge was plugged into a data source server that communicates with the federated data source manager functional component, and communicates with the added data sources via the data source cartridge;

receiving, by the plug-in framework functional component, the physical execution plan from the analytics component;

transferring, by the plug-in framework functional component, at least a part of the physical execution plan to the data source cartridge, wherein the data source cartridge is configured to generate a physical query to the added data source based on the part of the physical execution plan;

receiving, by the plug-in framework functional component, a result set from the data source cartridge, with the result set being a response from the added data source to the physical query, wherein the data source cartridge is used instead of a code generator, an execution engine, and one of the separate gateways of the federated data source manager functional component to provide the result set and wherein the federated data source manager functional component uses the separate gateways, the code generator, and the execution engine to provide result sets from the predetermined data sources; and

transferring, by the plug-in framework functional component, the result set to the analytics component to facilitate the generation of a response to the logical query.

7. The method of claim 6 further comprising:

transforming, by the plug-in framework functional component, at least a portion of the physical execution plan to a first Extensible Markup Language (XML) representation.

8. The method of claim 7 wherein the added data source is a Java data source and further comprising:

transforming the first XML representation of the physical execution plan to a Java-side execution plan.

9. The method of claim 6 wherein providing metadata received from the data source cartridge further comprises:

providing a feature list of the added data source.

10. The method of claim 6 wherein providing metadata received from the data source cartridge further comprises:

providing metadata of a new connection pool type not supported by the federated data source manager functional component and wherein the physical execution plan includes at least one query utilizing the new connection pool type.

11. A method, performed by a data source plug-in component executed by a data source server, with the data source plug-in component associated with an added data source that is not supported by native code of a federated data source manager, the method comprising:

outputting, by the data source plug-in component, metadata of the added data source to be used by the federated data source manager to construct a logical query including the added data source, wherein the federated data source manager includes separate gateways for each of predetermined data sources and wherein the data source plug-in component was plugged into the data source server that communicates with the federated data source manager, and communicates with the added data sources via the data source plug-in component;

receiving, by the data source plug-in component, part of a physical execution plan generated by the federated data source manager, with the part of the physical execution plan including at least one query to the added data source;

generating, by the data source plug-in component, a physical query to the added data source based on the part of the physical execution plan; and

returning, by the data source plug-in component, a result set to the federated data source manager in response to the physical query, wherein the data source plug-in component is used instead of a code generator, an execution engine, and one of the separate gateways of the federated data source manager to provide the result set and wherein the federated data source manager uses the separate gateways, the code generator, and the execution engine to provide result sets from the predetermined data sources.

12. The method of claim 11 wherein outputting metadata of the added data source further comprises:

outputting a feature list of the added data source.

13. The method of claim 11 further comprising:

outputting metadata of a new connection pool type not supported by the federated data source manager and wherein the physical execution plan includes at least one query utilizing the new connection pool type.

14. A method, performed by a connection pool plug-in component executed by a data source server, with the connection pool plug-in component associated with an added connection pool type that is not supported by native code of a federated data source manager, the method comprising:

outputting, by the connection pool plug-in component, metadata of the added connection pool type to be used by the federated data source manager to construct a logical query utilizing the added connection pool type, wherein the federated data source manager includes separate gateways for each of predetermined connection pool types and wherein the connection pool plug-in component was plugged into the data source server that communicates with the federated data source manager, and communicates with the added connection pool via the connection pool plug-in component;

receiving, by the connection pool plug-in component, part of a physical execution plan generated by the federated data source manager, with the part of the physical execution plan requiring the added connection pool type;

generating, by the connection pool plug-in component, a physical query using the added connection pool type and based on the part of the physical execution plan; and

returning to the federated data source manager, by the connection pool plug-in component, a result set using the added connection pool type, wherein the connection pool plug-in component is used instead of a code generator, an execution engine, and one of the separate gateways of the federated data source manager to provide the result set and wherein the federated data source mange uses the separate gateways, the code generator, and the execution engine to provide result sets from the predetermined connection pool types.

15. A computer system including one or more processors configured to execute federated data source manager program code, wherein the computer system is configured to receive a logical query to a federated database, wherein the federated database aggregates a predetermined set of predetermined data sources into a logical model and with the federated data source manager program code, when executed by the one or more processors, causing the computer system to perform the steps of:

adding a now data source to the federated database, wherein the federated data source manager program code includes separate gateways for each of the predetermined data sources;

plugging into a data source server that communicates with the federated data source manager program code, a new data source cartridge for the new data source;

retrieving metadata from a data source cartridge, with the data source cartridge configured to supply metadata of an added data source associated with the data source cartridge, where the added data source is not included in the predetermined set of predetermined data sources;

generating a physical execution plan, using at least a portion of the metadata retrieved from the data source cartridge, for the logical query with the physical execution plan including at least one query to the added data source;

providing at least a put of the physical execution plan to the data source cartridge to facilitate generating a physical query, by the data source cartridge, to the added data source to implement the part of the physical execution plan;

receiving, from the data source cartridge, a result set from the physical query to the added data source, wherein the data source cartridge is used instead of a code generator, an execution engine, and one of the separate gateways of the federated data source manager program code to provide the result set; and

using the code generator, the execution engine, and the separate gateways to provide result sets from the predetermined data sources,

wherein the federated data source manager program code does not include program code for generating a physical query to the added data source.

16. The computer system of claim 15 further configured to perform, when executing the federated data source manager program code, the step of:

transforming at least a portion of the physical execution plan to a first Extensible Markup Language (XML) representation.

17. The computer system of claim 16 wherein the added data source is a Java data source and with the computer system further configured to perform, when executing the federated data source manager program code, the step of:

transforming the first XML representation of the physical execution plan to a Java-side execution plan.

18. The computer system of claim 15 wherein retrieving metadata from the data source cartridge further comprises:

receiving a feature list of the added data source.

19. The computer system of claim 15 wherein retrieving metadata from the data source cartridge further comprises:

receiving metadata of a new connection pool type not supported by the federated data source manager program code and wherein the physical execution plan includes at least one query utilizing the new connection pool type.

20. A computer system including one or more processors configured to execute plug-in framework program code, wherein a federated data source manager functional component is configured to federate only a predetermined set of predetermined data sources into a federated database and includes an execution function component configured to generate physical queries to only data sources in the predetermined set and wherein the federated data source manager functional component includes an analytics component for creating a physical execution plan from a received logical query, with the plug-in framework program code, when executed by the one or more processors, causing the computer system to perform the steps of:

providing, to the analytics component of the federated data source manager functional component, metadata of an added data source received from a data source cartridge associated with the added data source, where the added data source is not included in the predetermined set of predetermined data sources and where the analytics component of the federated data source manager functional component is configured to utilize the metadata to generate the physical execution plan including the added data source, wherein the federated data source manager functional component includes separate gateways for each of the predetermined data sources and wherein the data source cartridge was plugged into a data source server that communicates with the federated data source manager functional component, and communicates with the added data sources via the data source cartridge;

receiving the physical execution plan from the analytics component;

transferring at least a part of the physical execution plan to the data source cartridge, wherein the data source cartridge is configured to generate a physical query to the added data source based on the part of the physical execution plan;

receiving a result set from the data source cartridge, with the result set being a response from the added data source to the physical query, wherein the data source cartridge is used instead of a code generator, an execution engine, and one of the separate gateways of the federated data source manager functional component to provide the result set and wherein the federated data source manager functional component uses the separate gateways, the code generator, and the execution engine to provide result sets from the predetermined data sources; and

transferring the result set to the analytics component to facilitate the generation of a response to the logical query.

21. The computer system of claim 20 further configured to perform, when executing the plug-in framework program code, the step of:

transforming at least a portion of the physical execution plan to a first Extensible Markup Language (XML) representation.

22. The computer system of claim 21 wherein the added data source is a Java data source and further comprising:

transforming the first XML representation of the physical execution plan to a Java-side execution plan.

23. The computer system of claim 20 wherein providing metadata received from the data source cartridge further comprises:

providing a feature list of the added data source.

24. The computer system of claim 20 wherein providing metadata received from the data source cartridge further comprises:

providing metadata of a new connection pool type not supported by the federated data source manager functional component and wherein the physical execution plan includes at least one query utilizing the new connection pool type.

25. A computer system including one or more processors configured to execute data source plug-in component program code, wherein the data source plug-in component program code is associated with an added data source that is not supported by native code of a federated data source manager and with the data source plug-in component program code, when executed by the one or more processors, causing the computer system to perform the steps of:

outputting metadata of the added data source to be used by the federated data source manager to construct a logical query including the added data source, wherein the federated data source manager includes separate gateways for each of predetermined data sources and wherein the data source plug-in component program code was plugged into a data source server that communicates with the federated data source manager, and communicates with the added data sources via the data source plug-in component program code;

receiving part of a physical execution plan generated by the federated data source manager, with the part of the physical execution plan including at least one query to the added data source;

generating a physical query to the added data source based on the part of the physical execution plan; and

returning a result set to the federated data source manager in response to the physical query, wherein the data source plug-in component program code is used instead of a code generator, an execution engine, and one of the separate gateways of the federated data source manager to provide the result set and wherein the federated data source manager uses the separate gateways, the code generator, and the execution engine to provide result sets from the predetermined data sources.

26. The computer system of claim 25 wherein outputting metadata of the added data source further comprises:

outputting a feature list of the added data source.

27. The computer system of claim 25 further configured to perform, when executing the data source plug-in component program code, the step of:

outputting metadata of a new connection pool type not supported by the federated data source manager and wherein the physical execution plan includes at least one query utilizing the new connection pool type.

28. A computer system including one or more processors configured to execute connection pool plug-in component program code, wherein the connection pool plug-in component is associated with an added connection pool type that is not supported by native code of a federated data source manager and with the connection pool plug-in component program code, when executed by the one or more processors, causing the computer system to perform the steps of:

outputting metadata of the added connection pool type to be used by the federated data source manager to construct a logical query utilizing the added connection pool type, wherein the federated data source manager includes separate gateways for each of predetermined data sources and wherein a data source cartridge was plugged into a data source server that communicates with the federated data source manager, and communicates with added data sources via the data source cartridge;

receiving part of a physical execution plan generated by the federated data source manager, with the part of the physical execution plan requiring the added connection pool type;

generating a physical query using the added connection pool type and based on the part of the physical execution plan; and

returning to the federated data source manager a result set using the added connection pool type, wherein the data source cartridge is used instead of a code generator, an execution engine, and one of the separate gateways of the federated data source manager to provide the result set and wherein the federated data source manager uses the separate gateways, the code generator, and the execution engine to provide result sets from the predetermined data sources.