Method and system for federated querying of data sources

• US 20050234889A1
• Filed: 12/29/2004
• Published: 10/20/2005
• Est. Priority Date: 05/25/2001
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Meir, Rannen, Schreiber, Zvi, Fox, Joshua
• Granted Patent: US 8,412,746 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/256 in federated or virtual databases

Abstract

A system for managing and querying a plurality of data sources, including an interface to a plurality of inter-related data sources of diverse types, each of the plurality of data sources having a schema that describes its data structure, a schema generator communicating with the interface, for generating a federated schema that describes the structures of the plurality of data sources and their inter-relationships, and for modifying the federated schema over time as the plurality of data sources undergo changes, and a query generator communicating with the schema generator, for generating a query for the federated schema. A method and a computer-readable storage medium are also described.

Claims

1. A system for managing and querying a plurality of data sources, comprising:

an interface to a plurality of inter-related data sources of diverse types, each of the plurality of data sources having a schema that describes its data structure;

a schema generator communicating with said interface, for generating a federated schema that describes the structures of the plurality of data sources and their inter-relationships, and for modifying the federated schema over time as the plurality of data sources undergo changes; and

a query generator communicating with said schema generator, for generating a query for the federated schema.

2. The system of claim 1 further comprising a query engine communicating with said interface, for executing a query to the federated schema and for returning query results.

3. The system of claim 2 wherein said query engine is a software application.

4. The system of claim 3 wherein said query engine is a component of a business intelligence tool.

5. The system of claim 3 wherein said query engine is a component of a business process management environment.

6. The system of claim 1 further comprising a semantic modeler communicating with said interface and with said schema generator, for mapping the plurality of data sources into a common ontology model using semantic mappings, and wherein said query generator generates a query formulated in terms of nomenclature of the common ontology model.

7. The system of claim 6 further comprising a view generator communicating with said semantic modeler, for generating a database view of the plurality of data sources, using the federated schema, wherein the database view is formulated in terms of nomenclature of the common ontology model, and wherein said query generator generates a query for the database view.

8. The system of claim 7 wherein the database view is formulated using SQL statements.

9. The system of claim 7 wherein the database view is formulated using XQuery statements.

10. The system of claim 7 wherein said view generator updates the database view when the common ontology model is changed.

11. The system of claim 6 further comprising a user manager communicating with said semantic modeler for managing user access privileges to the plurality of data sources, wherein user access privileges are expressed in terms of nomenclature of the common ontology model.

12. The system of claim 6 further comprising a data thesaurus communicating with said semantic modeler, for looking up the meaning of a data construct within the plurality of data sources, relative to the common ontology model.

13. The system of claim 12 wherein said data thesaurus traces the origin of a data construct within the plurality of data sources.

14. The system of claim 12 wherein said data thesaurus traces query results back to data sources.

15. The system of claim 6 wherein said query generator comprises a web-based user interface for creating and editing queries while browsing the common ontology model.

16. The system of claim 6 wherein the common ontology model includes classes, properties of classes, inheritances among classes, and business rules relating properties of classes, and wherein said query generator formulates a semantic query from an expression language that uses the classes, properties, inheritances and business rules.

17. The system of claim 16 further comprising a view generator communicating with said semantic modeler, for generating a database view of the plurality of data sources, using the federated schema, wherein the database view is formulated in terms an expression language that uses the classes, properties, inheritances and business rules, and wherein said query generator generates a query for the database view.

18. The system of claim 17 wherein said view generator automatically embeds business rules within view definitions.

19. The system of claim 17 wherein said view generator updates the database view when a relevant business rule is changed.

20. The system of claim 1 wherein the plurality of data sources of diverse types include at least one Microsoft SQL database and one XML document data source.

21. The system of claim 1 wherein the plurality of data sources of diverse types include at least one Microsoft SQL database and one IBM DB2 database.

22. The system of claim 1 wherein the plurality of data sources of diverse types include at least one XML document data source and one IBM DB2 database.

23. The system of claim 1 wherein said schema generator modifies the federated schema so as to make it appear that one or more of the plurality of data sources is removed.

24. The system of claim 1 wherein said schema generator modifies the federated schema when a new data source is added to the plurality of data sources.

25. The system of claim 24 wherein said schema generator automatically detects and scans new data sources.

26. The system of claim 1 wherein said schema generator modifies the federated schema when one of the plurality of data sources is modified.

27. A method for managing and querying a plurality of data sources, comprising:

generating a federated schema that describes the structures of a plurality of inter-related data sources and their inter-relationships, and for modifying the federated schema over time as the plurality of data sources undergo changes, wherein each of the plurality of data sources has a schema that describes its data structure; and

generating a query for the federated schema.

28. The method of claim 27 further comprising:

executing a query to the federated schema; and

returning query results.

29. The method of claim 28 wherein said executing is performed by a software application.

30. The method of claim 29 wherein said executing is performed by a business intelligence tool.

31. The method of claim 29 wherein said executing is performed within a business process management environment.

32. The method of claim 27 further comprising mapping the plurality of data sources into a common ontology model using semantic mappings, and wherein said generating a query formulates a query in terms of nomenclature of the common ontology model.

33. The method of claim 32 further comprising generating a database view of the plurality of data sources, using the federated schema, wherein the database view is formulated in terms of nomenclature of the common ontology model, and wherein said generating a query generates a query for the database view.

34. The method of claim 33 wherein the database view is formulated using SQL statements.

35. The method of claim 33 wherein the database view is formulated using XQuery statements.

36. The method of claim 33 wherein said generating a database view updates the database view when the common ontology model is changed.

37. The method of claim 32 further comprising managing user access privileges to the plurality of data sources, wherein user access privileges are expressed in terms of nomenclature of the common ontology model.

38. The method of claim 32 further comprising looking up the meaning of a data construct within the plurality of data sources, relative to the common ontology model.

39. The method of claim 38 wherein said looking up traces the origin of a data construct within the plurality of data sources.

40. The method of claim 38 wherein said looking up traces query results back to data sources.

41. The method of claim 32 wherein said generating a query employs a web-based user interface for creating and editing queries while browsing the common ontology model.

42. The method of claim 32 wherein the common ontology model includes classes, properties of classes, inheritances among classes, and business rules relating properties of classes, and wherein said generating a query formulates a query from an expression language that uses the classes, properties, inheritances and business rules.

43. The method of claim 42 further comprising generating a database view of the plurality of data sources, using the federated schema, wherein the database view is formulated in terms an expression language that uses the classes, properties, inheritances and business rules, and wherein said generating a query generates a query for the database view.

44. The method of claim 43 wherein said generating a database view automatically embeds business rules within view definitions.

45. The method of claim 43 wherein said generating a database view updates the database view when a relevant business rule is changed.

46. The method of claim 27 wherein the plurality of data sources of diverse types include at least one Microsoft SQL database and one XML document data source.

47. The method of claim 27 wherein the plurality of data sources of diverse types include at least one Microsoft SQL database and one IBM DB2 database.

48. The method of claim 27 wherein the plurality of data sources of diverse types include at least one XML document data source and one IBM DB2 database.

49. The method of claim 27 wherein said generating a federated schema comprises modifying the federated schema so as to make it appear that one or more of the plurality of data sources is removed.

50. The method of claim 27 wherein said generating a federated schema comprises modifying the federated schema when a new data source is added to the plurality of data sources.

51. The method of claim 50 further comprising automatically detecting and scanning new data sources.

52. The method of claim 27 wherein said generating a federated schema comprises modifying the federated schema when one of the plurality of data sources is modified.

53. A computer-readable storage medium storing program code for causing a computer to perform the steps of:

generating a federated schema that describes the structures of a plurality of inter-related data sources and their inter-relationships, and for F modifying the federated schema over time as the plurality of data sources undergo changes, wherein each of the plurality of data sources has a schema that describes its data structure; and

generating a query for the federated schema.