System and method for autonomously generating heterogeneous data source interoperability bridges based on semantic modeling derived from self adapting ontology

• US 20030172368A1
• Filed: 12/23/2002
• Published: 09/11/2003
• Est. Priority Date: 12/26/2001
• Status: Abandoned Application

• Current Assignee: Finsbury Solutions Ltd., Compass SPA (Mediobanca SPA)
• Original Assignee: Finsbury Solutions Ltd.
• Inventors: Reynolds, Ronald Joseph, Bohorquez, Yuri Adrian Tijerino, Bain, Mary Elizabeth, Rasmussen, Steven John, Lucky, David Eugene, Alumbaugh, Elizabeth
• Application Number:
  US10/329,153
• Publication Number:
  US 20030172368A1
• Time in Patent Office: Days
• US Class Current: 717/106
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

A system, including software components, that efficiently and dynamically analyzes changes to data sources, including application programs, within an integration environment and simultaneously re-codes dynamic adapters between the data sources is disclosed. The system also monitors at least two of said data sources to detect similarities within the data structures of said data sources and generates new dynamic adapters to integrate said at least two of said data sources. The system also provides real time error validation of dynamic adapters as well as performance optimization of newly created dynamic adapters that have been generated under changing environmental conditions.

Claims

1. A system connected to multiple heterogeneous data sources each having a data structure, said system monitoring at least one of said data structures, analyzing changes to said at least one of said data structure and providing for simultaneous re-coding of adapters between at least two of said multiple heterogeneous data sources.

2. The system of claim 1 including a system component for monitoring at least one data source and automatically detecting changes in the data structure of said data source.

9. The system of claim 1 further including error management components for automatically testing said recoded dynamic adapters before they are placed into operation.

10. The process of claim 2 further including the step of generating programming code automatically in response to said automatically detecting changes.

13. The system of claim 1 further including models that are jobs, applications, users, change specifications, schemas, applications, ontologies, App2App similarity maps, at least one Common Ontology and at least one database.

14. The system of claim 1 further including system managers for managing system-wide settings and data, schema managers for providing, storing, listing, and deleting schemas, user managers for managing users and their preferences, change specification managers for managing storage and retrieval of change specifications, job managers for managing jobs performing analysis or automation, task managers for managing and running scheduled tasks, ontology managers for mapping the access to and modification of the Common Ontology or other application ontologies, language managers for managing different programming languages in which the system can produce integration adapters.

15. The system of claim 14 wherein each said change specification represents the changes between two specific snapshots of a schema.

16. The system of claim 14 wherein a language manager allows a user to set preferences for delivery of language specific adapters.

17. The system of claim 1 further including an application ontology factory for mapping schemata of a plurality of data sources to the common ontology to produce data source specific ontologies;

an App2App Similarity Mapper for mapping a specific data source ontology to another data source ontology and producing a map of potential integration points between the two data sources;

an ontology editor functioning both as a manager and a factory; and

a Planner for producing an interactive integration plan between two disparate data sources based on the App2App similarity map.

18. The system of claim 17 wherein said ontology editor manages direct human interaction with the common ontology for validation, expansion and modification of said common ontology.

19. The system of claim 18 wherein said ontology editor provides a visual representation of the common ontology.

20. The system of claim 19 wherein said factories produce specific kinds of models.

21. The system of claim 20 wherein said factories manage persistence operations for said models set forth in claim 13.

22. The system of claim 1 further including (a) a Codegen Agent for interacting with a planner, a change specification manager, an App2App ontology factory and external data source-specific settings to generate and adapt integration code, and (b) a deployment agent for interacting with external data source environment elements and a Codegen Agent for deploying code in a self-adapting fashion.

23. The system of claim 22 wherein said Codegen Agent validates said deployed code.

24. The system of claim 22 wherein said components run on a backend server.

25. The system of claim 1 further including a desktop client running on users'"'"' or clients'"'"' desktops, said desktop capable of making requests of the system server components via system Proxies, receiving data from those requests, and presenting that data to the user, said desktop comprising an Application Context, a Schema Context, a change Specification context, a Report Generation Context, a Task List Context, an Admin Context, a User Administration context, a Notification context, an Application Ontology View context, an App2App Similarity Mapping Context, a Plan View context, a Language editor and a Code Browser context.

26. The system of claim 25 wherein said Application Context lists previously defined data sources and shows detailed information for the selected data source.

27. The system of claim 26 wherein the Application Context allows a user to add, modify or remove data source definitions.

28. The system of claim 25 wherein the Schema context lists previously collected schemas and shows detailed information for a selected schema.

29. The system of claim 25 wherein the Schema context shows detailed information for the selected schema and allows a user to add or remove schemas.

30. The system of claim 25 wherein the Change Specification Context lists the previously created Change Specifications and shows detailed information for the selected change specification.

31. The system of claim 25 wherein the Change Specification Context allows a user to add or remove change specifications.

32. The system of claim 25 wherein the Report Generation Context allows retrieval of previously saved reports.

33. The system of claim 25 wherein the Report Generation Context creates a new report from an existing schema or change specification.

34. The system of claim 25 wherein the Report Generation Context allows a user to save the current report.

35. The system of claim 25 wherein the Task List Context lists the pending/scheduled tasks for the current user and allows said user to add, modify or remove a task.

36. The system of claim 25 wherein the User Administration Context lists users of the system and allows an administrator user to set up new users and administer passwords.

37. The system of claim 26 wherein the Notification Context displays notifications and sets up notification preferences.

38. The system of claim 25 wherein the Application Ontology View Context lists application ontologies and displays application ontologies for browsing.

39. The system of claim 25 wherein the App2App Similarity Mapping Context lists App2App Similarity Maps and displays App2App Similarity Maps for browsing and user acceptance.

40. The system of claim 25 wherein the Plan View Context lists Integration Plans and displays Integration Plans for user browsing and acceptance.

41. The system of claim 25 wherein the Language Editor lists languages supported by the system and displays specific language settings for user browsing and preference selection.

42. The system of claim 25 wherein the Code Browser Context displays code in specific language for user browsing, user saving and user preference settings.

43. The system of claim 1 including a System Hub for providing clients with components that can be used to directly communicate with server components.

44. The system of claim 1 further including software processes comprising an Assessment Micro Agent, an App2App Similarity Mapper, a Planner, a Hub, and Error Validation and Code Generation components.

45. The system of claim 44 wherein said Assessment Micro Agent component comprises a Schema, Change Specification, a Task Manager and a Job Manager.

3. A system connected to multiple heterogeneous data sources each having a data structure, said system monitoring at least two of said data sources to detect similarities within the data structures of said data sources and generating new dynamic adapters to integrate said at least two of said data sources.

5. The process of claim 3 wherein said system monitors a plurality of dynamic adapters generated under changing computer environment conditions, said process including the steps of providing real time error validation of said dynamic adapters and performance optimization of at least one of said dynamic adapters.

6. The process of claim 5 including the step of using syntactic processes to automatically create adapter maintenance and support plans.

7. The process of claim 6 wherein the step of using syntactic processes occurs in an App2App Ontology Mapper and a Planner.

8. The process of claim 6 including the step of automatically checking for errors in said dynamic adapter.

11. The process of claim 6 further including the steps of dynamically detecting changes, including revisions in said at least one data source, analyzing said revisions, generating data structure mapping between heterogeneous data sources, validating errors, and executing appropriate adapter modifications.

12. The process of claim 11 further including the step determining an optimum update for the said dynamic adapters.

4. The process in a system within an integration environment for analyzing changes to multiple heterogeneous data sources each having a data structure and providing for simultaneous re-coding of dynamic adapters between said multiple heterogeneous data sources, including the steps of intelligently analyzing the conceptual relationships and alternative data mapping strategies between a plurality of said data structures by utilizing intelligent computer programs to analyze and adapt to structural, contextual and semantic differences between said multiple heterogeneous data sources.

46. The process of operating on two data sources within a system including other components than said two data sources, said other components including at least a Common Ontology library, including the steps of:

monitoring each of said data sources by an Assessment Micro Agent including a Schema Manager,

said Assessment Micro Agent creating an inventory of the data structures and functionalities of said data sources and making said inventory available to predetermined ones of said other components of said system,

said Assessment Micro Agent detecting a change in either of said data sources and notifying at least some of said other components of the change.

47. The process of claim 46 further including the step of an Application Ontology Factory accepting a data structure inventory from said Schema Manager and information provided from said Common Ontology library to produce data source ontologies.

48. The process of claim 47 including the further step of an App2App Similarity Mapper accepting the information in the data source ontologies to produce a similarity map between the two data sources.

49. The process of claim 48 including the further step of a Planner using the information contained in said similarity map to produce an integration plan.

50. The process of claim 49 including the further step of a CodeGen Agent accepting the information provided in the integration plan and using it to produce integration code.

51. The process of claim 50 including the further steps of validating said integration code by an Error Management Micro Agent and deploying said integration code between the two data sources.

52. The process of claim 46 including the further step of the Schema Manager of said Assessment Micro Agent reading the data structure stored in a data source to produce a schema that is placed into a memory model.

53. The process of claim 52 including the steps of the Schema Manager collecting data source information, data source driver information, table names, table types, indexes, foreign keys, column names, column data types, column precision, column nullability, primary key designation, view definitions, synonym and alias references, and remarks stored in the database schema and providing said collected information to predetermined ones of said other components.

54. The process of claim 46 including the further steps of the Assessment Micro Agent, in response to a change in a monitored data source, detecting alterations including new information in the database structure of said data source and analyzing said change by comparing said new information of said alteration to data stored in the Schema Manager.

55. The process of claim 54 wherein said last named step is performed by the Change Specification Manager comparing one historical view of the schema for one data source to another historical view of said schema.

56. An Assessment Micro Agent comprising a plurality of components including:

a Schema—

Manager connected to at least one data source for analyzing said at least one data source and extracting a meta-data model in the form of a schema, storing said schema and providing an interface to certain of said plurality of components for retrieving the schema;

a Change Specification Manager for performing an analysis of what is different between two different versions of a data source by comparing the schemas associated with each version and presenting the change specification file to a user in a structured manner with specific information indicating changes in the schemas;

a Task scheduler for allowing a user to schedule tasks; and

a Notification Manager for providing an interface in which users can define notifications at several levels of granularity.

57. The Assessment Micro Agent of claim 56 wherein said levels of granularity include setting up notifications on the complete file of the change specifications or on filtered views of said files according to user preferences.

58. The Assessment Micro Agent of claim 56 wherein the Notification Manager can send notifications via standard mediums such as email, pager or PDAs according to user preferences.

59. The Assessment Micro Agent of claim 56 wherein the tasks include the generation of schemas through the Schema Manager and the generation of change specifications through the Change Specification Manager.

60. The Assessment Micro Agent of claim 56 further including the functions of monitoring connectivity between the Assessment Micro Agent and said data sources, managing the schema monitoring, retrieving change specifications, sending system-level notifications and user notifications, and allowing a user to create filtered views of changes according to one or more user preferences.

61. The process of operating an Application Ontology Factory including the steps of:

converting the schema obtained from the Schema Manager component of the Assessment Micro Agent into a language compatible to the Common Ontology;

mapping schema element identifiers to a WordNet to extract at least one of the senses of said elements;

using said senses to extract all possible Common Ontology concept hierarchies to which the element might be a top-most specialization;

assigning each concept hierarchy a confidence factor;

merging said concept hierarchies to produce a micro-theory including each of said senses.

62. The process of claim 61 wherein a schema element is associated with one or more concept hierarchies.

63. The process of claim 62 wherein each concept hierarchy has an independent confidence factor.

64. In an artificial intelligence system connected to multiple heterogeneous data sources for generating new dynamic adapters to integrate changes in at least two of said data sources, the process of describing a schema using the syntax of the Common Ontology language.

65. In a system for automatically re-coding interfaces between heterogeneous data sources the process of monitoring changes in a monitored data source, analyzing the exact nature of the change, evaluating alternative data mapping possibilities, and adjusting the existing dynamic adapter integration code structures to address the changes.

66. The process of claim 65 including the step of using synonym relations for lexical level mapping by computing lexical proximity of elements in the schemas of the data sources.

67. The process of claim 65 including the step of finding semantical proximity by using hypernym relationships.

68. The process of claim 65 including the step of using computing the closeness of data values on mapped schema elements.

69. In a system for automatically generating dynamic adapters between heterogeneous data sources the process of monitoring changes in a monitored data source using pattern matching, said process including the steps of:

generating a data source to ontology mapping for each data source being mapped by evaluating the mathematical probabilities of lexical and semantic relationships between schema entities and ontology concepts;

determining lexical closeness between the data source ontology and Common Ontology concepts using synonym relationships;

determining mathematical closeness of semantic relationships in the form of hypernyms; and

determining confidence factors based on the mathematical probability of said data source ontology and said Common Ontology being lexically and semantically close.

70. The process of claim 69 including the further steps of:

comparing the data source ontologies of the monitored data sources to determine common concepts;

mapping a data source ontology to another data source ontology using synonym and hypernym relationships;

extracting a sample of data element values from each said data sources and comparing said data element values to determine mathematical closeness;

validating expected data values for said data source ontology mappings;

composing and decomposing semantic relationships between target and source data source ontology elements; and

uniting semantically similar schema elements into new ontology concepts.

71. The process of claim 70 wherein the step of validating mappings using expected data values includes the step of validating said closeness by performing pattern matching on the data values of one data source data element and another data source data element by determining how close data values for said elements are.

72. The process of claim 71 including the step of using pattern-matching to normalize data properties of the data structures of the data sources including data type and data length.

73. The process of claim 70 wherein the step of composing semantic relationships includes the steps of comparing data values of data source data structure elements and deriving semantic similarity thereof based on semantic proximity of one data source'"'"'s data structure elements to another data source'"'"'s data structure elements.

74. The process of claim 70 wherein the step of decomposing semantic relationships includes the steps of:

determining that two data structure elements are similar;

determining that one of said data structures has data elements with no associated functional relationship and that said other data structure element has a functional relationship with other data structure elements;

determining whether said data elements display any similarity with said other data structure elements.

75. The process of claim 70 wherein the step of uniting data structure elements to form a new concept in the Common Ontology includes the step of mapping two or more different data structure elements from a data source to another data source by determining whether the mapped-to concept in the Common Ontology is the most specialized concept of a concept hierarchy in the Common Ontology and has no children concept, and adding said data structure as a concept to the Common Ontology.

76. In a system for automatically generating dynamic adapters between heterogeneous data sources, a Planner receiving the change specification file created by the Change Specification Manager and developing and logically testing an ordered dynamic adapter development plan.

109. The Planner of claim 76, said Planner being a software component for performing the process steps of (a) using a planning engine to evaluate confidence factors determined by an App2App Similarity Mapper and selecting higher confidence factors as planning goals and (b) determining the required data transformation steps that need to occur in order to accomplish said goals.

110. The Planner of claim 109 wherein the mappings having a confidence factor of 100% are provided to a user as planning goals with high degree of confidence and mappings with less than 100% confidence factors produce a plurality of alternative mapping goals.

111. The Planner of claim 110 including a software process responsive to said planning goals to produce the required data transformation steps to accomplish said planning goals.

77. In a system for automatically generating dynamic adapters between heterogeneous data sources, a Planner receiving a similarity map file created by an App2App Similarity Mapper and developing and logically testing an ordered dynamic adapter development plan.

78. The Planner of claim 77, said Planner being a software component for performing the process steps of (a) using a planning engine to evaluate confidence factors determined by an App2App Similarity Mapper and selecting higher confidence factors as planning goals and (b) determining the required data transformation steps that need to occur in order to accomplish said goals.

79. The Planner of claim 78 wherein the mappings having a confidence factor of 100% are provided to a user as planning goals with high degree of confidence and mappings with less than 100% confidence factors produce a plurality of alternative mapping goals.

80. The Planner of claim 79 including a software process responsive to said planning goals to produce the required data transformation steps to accomplish said planning goals.

81. An App2App Ontology Mapper for producing data mapping between schema elements, said mappings having confidence factors, said App2App Ontology Mapper including a software process for detecting that said mapping is accomplished by a lexical, semantic, expected data value, composition or decomposition process and, responsive to any such detecting, increasing said confidence factor.

82. An App2App Ontology Mapper for producing data mapping between schema elements, said mappings having confidence factors, said App2App Ontology Mapper including a software process for detecting that said mapping is refuted by a lexical, semantic, expected data value, composition or decomposition process and, responsive to any such detecting, lowering said confidence factor.

83. An App2App Ontology Mapper for producing data mappings between schema elements, said mappings having confidence factors, said App2App Ontology Mapper including a software process for assigning a lower confidence factor to mappings accomplished by lexical similarity than to mappings accomplished by lexical similarity plus semantic mapping.

84. An App2App Ontology Mapper for producing data mappings between schema elements, said mappings having confidence factors, said App2App Ontology Mapper including a software process for assigning a lower confidence factor to mappings accomplished by semantic mapping than to mappings accomplished by semantic mapping and expected data value mapping.

85. In a system for generating dynamic adapters between changed data sources, a process for generating dynamic adapters including the steps of:

after an integration plan between two data sources has been generated, an Assessment Micro Agent determining that one of said data source'"'"'s data structure has changed and, in response to said detecting, informing a Planner software component to generate a new plan if the previously generated plan has been affected by said change;

creating a Change Specification File that describes said changes that occurred;

discovering which schema elements of said dynamic adapter have changed;

mapping the affected schema elements into the existing data source ontology;

performing lexical and semantic mapping on the affected schema elements to find new associations with said data source ontology;

in response to finding said new associations, validating said new associations; and

attempting to find new mappings for the affected elements.

86. The process of claim 85 wherein said attempting to find new mappings is accomplished using an expected data value process.

87. The process of claim 85 including the further step of in response to finding no said mappings, attempting to find new mappings using composition and decomposition processes.

88. The process of claim 85 including the step of producing a new map and presenting said new map to a user.

89. The process of claim 88 including the step of detecting an indication that said user accepts said new map and, in response to said detecting of said indication, providing the map to the Planner.

90. The process of claim 89 wherein said Planner generates the new plan, said plan having confidence factors associated therewith.

91. In a system for generating revised dynamic adapters between changed data sources, a process for revising said adapters including the steps of:

a Planner presenting an integration plan approved by a user as input to a CodeGen Agent;

said CodeGen Agent executing the development of new adapters by reparsing said integration plan into a user-selected programming language.

92. The process of claim 91 wherein said reparsing is accomplished using a template file that contains transformation instructions to translate each integration operation into compilation-ready source code for the selected adapter language.

93. In a system for generating new dynamic adapters between data sources, a process for generating said adapters including the steps of:

a Planner presenting as input to a CodeGen Agent an integration plan approved by a user, said integration plan including an indication of a use-selected programming language;

said CodeGen Agent executing the development of new adapters by producing programming instructions to accomplish the integration plan in the user-elected programming language.

94. For use in a system for generating new dynamic adapters between data sources, an Error Management Micro Agent coupled to a Planner and accepting the output from said Planner to determine and categorize program errors and remediation plans.

95. The Error Management Micro Agent of claim 94 including a software process capable of detecting errors in one or more of the group consisting of generated code, data extraction, data aggregation and data insertion.

96. The Error Management Micro Agent of claim 95 wherein said detecting errors in said generated code is accomplished by using compiler and script verification technology.

97. The Error Management Micro Agent of claim 95 wherein detecting errors in data extraction, data aggregation and data insertion is accomplished by detecting one or more errors in the logical correctness of the generated code.

98. The Error Management Agent of claim 97 wherein the step of detecting one or more errors in the logical correctness of the code is accomplished by (a) use of a database emulator to emulate database tasks and, (b) comparing the results of the emulations against said plan presented by said Planner.

99. A system for automatically re-coding interfaces between heterogeneous data sources comprising:

means for monitoring modifications made to a data source existing within an integration environment, wherein the environment contains multiple heterogeneous data sources,

means for analyzing said modifications,

means for formulating a set of potential ontological mappings between heterogeneous data sources,

means for providing interoperability code structures between heterogeneous data sources.

100. The system of claim 99, wherein the system is additionally comprised of a means for error detection.

101. A system for automatically re-coding interfaces between heterogeneous data sources comprising:

means for monitoring and analyzing modification made to a data source existing within an integration environment, wherein the environment contains multiple heterogeneous data sources;

means for formulating a set of potential ontological mappings between heterogeneous data sources and providing interoperability code structures between data sources.

102. In a system for automatically generating dynamic adapters between heterogeneous data sources the process of generating a new adapter, said process including the steps of:

generating a data source to ontology mapping for each data source being mapped by evaluating the mathematical probabilities of lexical and semantic relationships between schema entities and ontology concepts;

determining lexical closeness between the data source ontology and Common Ontology concepts using synonym relationships;

determining mathematical closeness of semantic relationships in the form of hypernyms;

determining confidence factors based on the mathematical probability of said data source ontology and said Common Ontology being lexically and semantically close.

103. The process of claim 102 including the further steps of:

comparing the data source ontologies of the monitored data sources to determine common concepts;

mapping a data source ontology to another data source ontology using synonym and hypernym relationships;

extracting a sample of data element values from each said data sources and comparing said data element values to determine mathematical closeness;

validating expected data values for said data source ontology mappings;

composing and decomposing semantic relationships between target and source data source ontology elements; and

uniting semantically similar schema elements into new ontology concepts.

104. The process of claim 103 wherein the step of validating mappings using expected data values includes the step of validating said closeness by performing pattern matching on the data values of one data source data element and another data source data element by determining how close data values for said elements are.

105. The process of claim 104 including the step of using pattern-matching to normalize data properties of the data structures of the data sources including data type and data length.

106. The process of claim 103 wherein the step of composing semantic relationships includes the steps of comparing data values of data source data structure elements and deriving semantic similarity thereof based on semantic proximity of one data source'"'"'s data structure elements to another data source'"'"'s data structure.

107. The process of claim 103 wherein the step of decomposing semantic relationships includes the steps of:

determining that two data structure elements are similar;

determining that one of said data structures has data elements with no associated functional relationship and that said other data structure element has a functional relationship with other data structure elements;

determining whether said data elements display any similarity with said other data structure elements.

108. The process of claim 103 wherein the step of uniting data structure elements to form a new concept in the Common Ontology includes the step of mapping two or more different data structure elements from a data source to another data source by determining whether the mapped-to concept in the Common Ontology is the most specialized concept of a concept hierarchy in the Common Ontology and has no children concept, and adding said data structure as a concept to the Common Ontology.

112. In a system for generating dynamic adapters between two data sources, a process for developing dynamic adapters including the steps of:

before an integration plan between said two data sources has been generated, an App2App Similarity Mapper determining the similarities between said two data sources and informing a Planner software component to generate a new plan, said App2App Similarity Mapper performing at least the steps of;

creating an App2App similarity map that describes said similarities;

mapping the schema elements affected by said similarities to an existing data source ontology;

performing lexical and semantic mapping on the affected schema elements to find new associations with said data source ontology;

in response to finding said new associations, validating said new associations; and

attempting to find new mappings for the affected elements.

113. The process of claim 112 wherein said attempting to find new mappings is accomplished using an expected data value process.

114. The process of claim 112 including the further step of in response to finding no said mappings, attempting to find new mappings using composition and decomposition processes.

115. The process of claim 112 including the step of producing a new map and presenting said new map to a user.

116. The process of claim 115 including the step of detecting an indication that said user accepts said new map and, in response to said detecting of said indication, providing the map to the Planner.

117. The process of claim 116 wherein said Planner generates the new plan, said plan having confidence factors associated therewith.

118. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform in a system within an integration environment for analyzing changes to multiple heterogeneous data sources each having a data structure and providing for simultaneous re-coding of dynamic adapters between said multiple heterogeneous data sources, the process comprising the step of intelligently analyzing the conceptual relationships and alternative data mapping strategies between a plurality of said data structures by utilizing intelligent computer programs to analyze and adapt to structural, contextual and semantic differences between said multiple heterogeneous data sources.

119. The one or more processor readable storage devices of claim 118 wherein said system monitors a plurality of dynamic adapters generated under changing computer environment conditions where said process includes the further steps of providing real time error validation of said dynamic adapters and performance optimization of at least one of said dynamic adapters.

120. The one or more processor readable storage devices of claim 119 where said process includes the further step of using syntactic processes to automatically create adapter maintenance and support plans.

121. The one or more processor readable storage devices of claim 120 where said process includes the further step of using syntactic processes occurs in an App2App Ontology Mapper and a Planner.

122. The one or more processor readable storage devices of claim 121 where said process includes the further step of automatically checking for errors in said dynamic adapter.

123. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform a process of operating on two data sources within a system including other components than said two data sources, said other components including at least a Common Ontology library, the process comprising the steps of:

monitoring each of said data sources by an Assessment Micro Agent including a Schema Manager;

said Assessment Micro Agent creating an inventory of the data structures and functionalities of said data sources and making said inventory available to predetermined ones of said other components of said system;

said Assessment Micro Agent detecting a change in either of said data sources and notifying at least some of said other components of the change.

124. The one or more processor readable storage devices of claim 123 where said process includes the further step of an Application Ontology Factory accepting a data structure inventory from said Schema Manager and information provided from said Common Ontology library to produce data source ontologies.

125. The one or more processor readable storage devices of claim 124 where said process includes the further step of an App2App Similarity Mapper accepting the information in the data source ontologies to produce a similarity map between the two data sources.

126. The one or more processor readable storage devices of claim 125 where said process includes the further step of a Planner using the information contained in said similarity map to produce an integration plan.

127. The one or more processor readable storage devices of claim 126 where said process includes the further step of a CodeGen Agent accepting the information provided in the integration plan and using it to produce integration code.

128. The one or more processor readable storage devices of claim 127 where said process includes the further step of validating said integration code by an Error Management Micro Agent and deploying said integration code between the two data sources.

129. The one or more processor readable storage devices of claim 123 where said process includes the further step of the Schema Manager of said Assessment Micro Agent reading the data structure stored in a data source to produce a schema that is placed into a memory model.

130. The one or more processor readable storage devices of claim 129 where said process includes the further step of the Schema Manager collecting data source information, data source driver information, table names, table types, indexes, foreign keys, column names, column data types, column precision, column nullability, primary key designation, view definitions, synonym and alias references, and remarks stored in the database schema and providing said collected information to predetermined ones of said other components.

131. The one or more processor readable storage devices of claim 123 where said process includes the further step of the Assessment Micro Agent, in response to a change in a monitored data source, detecting alterations including new information in the database structure of said data source and analyzing said change by comparing said new information of said alteration to data stored in the Schema Manager.

132. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform a process of operating an Application Ontology Factory, the process comprising the steps of:

converting the schema obtained from the Schema Manager component of the Assessment Micro Agent into a language compatible to the Common Ontology;

mapping schema element identifiers to a WordNet to extract at least one of the senses of said elements;

using said senses to extract all possible Common Ontology concept hierarchies to which the element might be a top-most specialization;

assigning each concept hierarchy a confidence factor;

merging said concept hierarchies to produce a micro-theory including each of said senses.

133. The one or more processor readable storage devices of claim 132 wherein schema element is associated with one or more concept hierarchies.

134. The one or more processor readable storage devices of claim 133 wherein each concept hierarchy has an independent confidence factor.

135. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform a process, in an artificial intelligence system connected to multiple heterogeneous data sources for generating new dynamic adapters to integrate changes in at least two of said data sources, the process of describing a schema using the syntax of the Common Ontology language.

136. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform a process, in a system for automatically recoding interfaces between heterogeneous data sources, the process comprising the step of monitoring changes in a monitored data source, analyzing the exact nature of the change, evaluating alternative data mapping possibilities, and adjusting the existing dynamic adapter integration code structures to address the changes.

137. The one or more processor readable storage devices of claim 136 where said process includes the further step of using synonym relations for lexical level mapping by computing lexical proximity of elements in the schemas of the data sources.

138. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform, in a system for automatically generating dynamic adapters between heterogeneous data sources, the process of monitoring changes in a monitored data source using pattern matching, the process comprising the steps of:

generating a data source to ontology mapping for each data source being mapped by evaluating the mathematical probabilities of lexical and semantic relationships between schema entities and ontology concepts;

determining lexical closeness between the data source ontology and Common Ontology concepts using synonym relationships;

determining mathematical closeness of semantic relationships in the form of hypernyms; and

determining confidence factors based on the mathematical probability of said data source ontology and said Common Ontology being lexically and semantically close.

139. The one or more processor readable storage devices of claim 138 where said process includes the further steps of:

comparing the data source ontologies of the monitored data sources to determine common concepts;

mapping a data source ontology to another data source ontology using synonym and hypernym relationships;

extracting a sample of data element values from each said data sources and comparing said data element values to determine mathematical closeness;

validating expected data values for said data source ontology mappings;

composing and decomposing semantic relationships between target and source data source ontology elements; and

uniting semantically similar schema elements into new ontology concepts.

140. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform a process in a system for automatically generating dynamic adapters between heterogeneous data sources, the process comprising the step of a Planner receiving the change specification file created by the Change Specification Manager and developing and logically testing an ordered dynamic adapter development plan.

141. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform a process, in a system for automatically generating dynamic adapters between heterogeneous data sources, the process comprising the step of a Planner receiving a similarity map file created by an App2App Similarity Mapper and developing and logically testing an ordered dynamic adapter development plan.

142. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform a process in a system for generating dynamic adapters between changed data sources, said process for generating dynamic adapters including the steps of:

after an integration plan between two data sources has been generated, an Assessment Micro Agent determining that one of said data source'"'"'s data structure has changed and, in response to said detecting, informing a Planner software component to generate a new plan if the previously generated plan has been affected by said change;

creating a Change Specification File that describes said changes that occurred;

discovering which schema elements of said dynamic adapter have changed;

mapping the affected schema elements into the existing data source ontology;

performing lexical and semantic mapping on the affected schema elements to find new associations with said data source ontology;

in response to finding said new associations, validating said new associations; and

attempting to find new mappings for the affected elements.

143. The one or more processor readable storage devices of claim 142 wherein said attempting to find new mappings is accomplished using an expected data value process.

144. The one or more processor readable storage devices of claim 142 where said process includes the further step of in response to finding no said mappings, attempting to find new mappings using composition and decomposition processes.

145. The one or more processor readable storage devices of claim 142 where said process includes the further step of producing a new map and presenting said new map to a user.

146. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform, in a system for generating revised dynamic adapters between changed data sources, a process for revising said adapters the process comprising the steps of:

a Planner presenting an integration plan approved by a user as input to a CodeGen Agent;

said CodeGen Agent executing the development of new adapters by reparsing said integration plan into a user-selected programming language.

147. The one or more processor readable storage devices of claim 146 wherein said reparsing is accomplished using a template file that contains transformation instructions to translate each integration operation into compilation-ready source code for the selected adapter language.

148. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform, in a system for generating new dynamic adapters between data sources, a process for generating said adapters, the process comprising the steps of:

a Planner presenting as input to a CodeGen Agent an integration plan approved by a user, said integration plan including an indication of a use-selected programming language;

said CodeGen Agent executing the development of new adapters by producing programming instructions to accomplish the integration plan in the user-elected programming language.

149. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform, in a system for automatically generating dynamic adapters between heterogeneous data sources the process of generating a new adapter, the process comprising the steps of:

generating a data source to ontology mapping for each data source being mapped by evaluating the mathematical probabilities of lexical and semantic relationships between schema entities and ontology concepts;

determining lexical closeness between the data source ontology and Common Ontology concepts using synonym relationships;

determining mathematical closeness of semantic relationships in the form of hypernyms;

determining confidence factors based on the mathematical probability of said data source ontology and said Common Ontology being lexically and semantically close.

150. The one or more processor readable storage devices of claim 149 where said process includes the further steps of:

comparing the data source ontologies of the monitored data sources to determine common concepts;

mapping a data source ontology to another data source ontology using synonym and hypernym relationships;

extracting a sample of data element values from each said data sources and comparing said data element values to determine mathematical closeness;

validating expected data values for said data source ontology mappings;

composing and decomposing semantic relationships between target and source data source ontology elements; and

uniting semantically similar schema elements into new ontology concepts.

151. One or more processor readable storage devices having processor readable code embodied on said processor readable storage devices, said processor readable code for programming one or more processors to perform, in a system for generating dynamic adapters between two data sources, a process for developing dynamic adapters, the process comprising the steps of:

before an integration plan between said two data sources has been generated, an App2App Similarity Mapper determining the similarities between said two data sources and informing a Planner software component to generate a new plan, said App2App Similarity Mapper performing at least the steps of;

creating an App2App similarity map that describes said similarities;

mapping the schema elements affected by said similarities to an existing data source ontology;

performing lexical and semantic mapping on the affected schema elements to find new associations with said data source ontology;

in response to finding said new associations, validating said new associations; and

attempting to find new mappings for the affected elements.

152. The one or more processor readable storage devices of claim 151 wherein said attempting to find new mappings is accomplished using an expected data value process.

153. The one or more processor readable storage devices of claim 151 where said process includes the further step of, in response to finding no said mappings, attempting to find new mappings using composition and decomposition processes.

154. A process of managing revision in a data source including the steps of:

connecting an Assessment Micro Agent to a data source;

using the Schema Manager, extracting information about the data source;

using the Schema Manager, building a schema of the data source from at least some of said extracted information; and

presenting the schema to a user.

155. The process of claim 154 including the additional steps of:

the user selecting schema elements of interest to the user and creating a filtered view thereof; and

the user using the Task Manager to schedule frequency for generating schema specifications.

156. The process of claim 155 including the additional steps of:

the Change Specification Manager identifying a change in any of the selected schema elements during running of said data source; and

in response to said identifying, informing the user of said detected change.

157. The process of claim 154 wherein the step of collecting information includes the step of collecting data source information, connectivity driver information, table names and types, indexes, primary keys, foreign keys, column names and types, column precision, view definitions, synonym and alias references, and remarks stored in a database schema.