System and method for ontology induction through statistical profiling and reference schema matching

• US 10,620,924 B2
• Filed: 08/22/2017
• Issued: 04/14/2020
• Est. Priority Date: 08/22/2016
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Stojanovic, Alexander Sasha, Namarvar, Hassan Heidari, Allan, David, Seetharaman, Ganesh
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US15/683,559
• Publication Number:
  US 20180052870A1
• Time in Patent Office: 966 Days
• Field of Search: 707803
• CPC Class Codes: G06F 16/144 Query formulation
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 8/10 Requirements analysis; Specification techniques
• CPC Class Codes: G06F 8/34 Graphical or visual programming
• CPC Class Codes: G06F 8/433 Dependency analysis; Data or control flow analysis
• CPC Class Codes: G06F 8/4452 Software pipelining
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 5/046 Forward inferencing; Production systems
• CPC Class Codes: G06Q 10/0637 : Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals

Abstract

In accordance with various embodiments, described herein is a system (Data Artificial Intelligence system, Data AI system), for use with a data integration or other computing environment, that leverages machine learning (ML, DataFlow Machine Learning, DFML), for use in managing a flow of data (dataflow, DF), and building complex dataflow software applications (dataflow applications, pipelines). In accordance with an embodiment, the system can perform an ontology analysis of a schema definition, to determine the types of data, and datasets or entities, associated with that schema; and generate, or update, a model from a reference schema that includes an ontology defined based on relationships between datasets or entities, and their attributes. A reference HUB including one or more schemas can be used to analyze data flows, and further classify or make recommendations such as, for example, transformations enrichments, filtering, or cross-entity data fusion of an input data.

Claims

1. A method for use with a data integration or other computing environment comprising:

receiving input defining one or more schemas;

accessing the one or more schemas to obtain one or more entity definitions associated with entities provided by the reference of one or more schemas;

generating a sample data for the one or more entities from the one or more schemas;

profiling the sample data to determine one or more metrics associated with the sample data;

generating one or more rules based on the entity definitions; and

generating a functional type system based on the generated one or more rules, for use in processing a data input.

2. The method of claim 1, wherein the one or more rules includes data rules that are defined in terms of profiled data attribute, or composite value metrics;

relationship rules that define associations across entities and attribute vectors; and

complex rules that can be derived through a combination of data and relationship rules.

3. The method of claim 1, wherein the one or more schemas are provided in a reference HUB.

4. The method of claim 1, wherein the functional type system is persisted to a knowledge source.

5. The method of claim 4, wherein the knowledge source is a system HUB.

6. The method of claim 1, wherein the one or more schemas operate as a reference ontology, for use in type-tagging, comparing, classifying, or otherwise evaluating a metadata schema or ontology provided by registered HUBs.

7. The method of claim 1, wherein the method is performed in a cloud or cloud-based computing environment.

8. A system for ontology analysis of a schema definition for use with a data integration or other computing environment, comprising:

one or more processors operable to;

receiving input defining one or more schemas;

accessing the one or more schemas to obtain one or more entity definitions associated with entities provided by the reference of one or more schemas;

generating a sample data for the one or more entities from the one or more schemas;

profiling the sample data to determine one or more metrics associated with the sample data;

generating one or more rules based on the entity definitions; and

generating a functional type system based on the generated one or more rules, for use in processing a data input.

9. The system of claim 8, wherein the one or more rules includes data rules that are defined in terms of profiled data attribute, or composite value metrics;

relationship rules that define associations across entities and attribute vectors; and

complex rules that can be derived through a combination of data and relationship rules.

10. The system of claim 8, wherein the one or more schemas are provided in a reference HUB.

11. The system of claim 8, wherein the functional type system is persisted to a knowledge source.

12. The system of claim 11, wherein the knowledge source is a system HUB.

13. The system of claim 8, wherein the one or more schemas operate as a reference ontology, for use in type-tagging, comparing, classifying, or otherwise evaluating a metadata schema or ontology provided by registered HUBs.

14. The system of claim 8, wherein the system is provided in a cloud or cloud-based computing environment.

15. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by one or more computers cause the one or more computers to perform a method comprising:

receiving input defining one or more schemas;

accessing the one or more schemas to obtain one or more entity definitions associated with entities provided by the reference of one or more schemas;

generating a sample data for the one or more entities from the one or more schemas;

profiling the sample data to determine one or more metrics associated with the sample data;

generating one or more rules based on the entity definitions; and

generating a functional type system based on the generated one or more rules, for use in processing a data input.

16. The non-transitory computer readable storage medium of claim 15, wherein the one or more rules includes data rules that are defined in terms of profiled data attribute, or composite value metrics;

relationship rules that define associations across entities and attribute vectors; and

complex rules that can be derived through a combination of data and relationship rules.

17. The non-transitory computer readable storage medium of claim 15, wherein the one or more schemas are provided in a reference HUB.

18. The non-transitory computer readable storage medium of claim 15, wherein the functional type system is persisted to a knowledge source.

19. The non-transitory computer readable storage medium of claim 18, wherein the knowledge source is a system HUB.

20. The non-transitory computer readable storage medium of claim 15, wherein the one or more schemas operate as a reference ontology, for use in type-tagging, comparing, classifying, or otherwise evaluating a metadata schema or ontology provided by registered HUBs.