System, method and program product to estimate cost of integrating and utilizing heterogeneous data sources

• US 8,145,579 B2
• Filed: 09/01/2010
• Issued: 03/27/2012
• Est. Priority Date: 05/17/2005
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Iqbal, Mickey
• Primary Examiner(s): Hayes, John
• Assistant Examiner(s): Chen, George
• Application Number:
  US12/873,903
• Publication Number:
  US 20110040703A1
• Time in Patent Office: 573 Days
• Field of Search: None
• US Class Current: 705/400
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06Q 30/0283 Price estimation or determination

Abstract

System, method and program product for estimating a cost of reconciling heterogeneous data sources. A transition cost for integrating together a first program to identify semantic conflicts, a second program to classify semantic conflicts and a third program to reconcile semantic conflicts is estimated. A steady state cost for managing and maintaining the integrated first, second and third programs is estimated. Another system, method and program product for estimating a cost of integrating heterogeneous data sources. A steady state cost of managing and maintaining a first program which identifies semantic conflicts between a cross data source query and schema elements in a data source is estimated. A steady state cost of managing and maintaining a second program which classifies semantic conflicts between the cross data source query and schema elements in the data source is estimated. A steady state cost of managing and maintaining a third program which reconciles semantic conflicts between the cross data source query and schema elements in the data source is estimated.

Claims

1. A method for estimating a labor cost to reconcile semantic conflicts between data schema terms used in different data sources, the method comprising the steps of:

a computer estimating a labor cost for mapping, to shared ontology terms, respective pairs of the data schema terms having semantic conflicts with each other, the computer system estimating the labor cost based on at least five of the following;

(a) a number of the data sources that contain the data schema terms having the semantic conflicts, (b) an approximate number of the data schema terms in each of the data sources, (c) an approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) an approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) an approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) an approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) an approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms; and

the computer displaying on a monitor the estimated labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other.

2. The method of claim 1 wherein the computer estimates the labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other, based on at least six of the following:

(a) the number of the data sources that contain the data schema terms having the semantic conflicts, (b) the approximate number of the data schema terms in each of the data sources, (c) the approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) the approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) the approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) the approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) the approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms.

3. The method of claim 1 wherein the computer estimates the labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other, based on all of the following:

(a) the number of the data sources that contain the data schema terms having the semantic conflicts, (b) the approximate number of the data schema terms in each of the data sources, (c) the approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) the approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) the approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) the approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) the approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms.

4. The method of claim 1 further comprising the step of:

the computer estimating a labor cost to identify mistakes in the mappings from a plurality of pairs of the data schema terms to a respective plurality of the shared ontology terms, based on differences in human language text associated with the data schema terms in each of the pairs, and determine correct mappings of the data schema terms in each of the pairs to shared ontology terms.

5. The method of claim 1 further comprising the step of:

the computer estimating a labor cost to create an information wrapper to map terms in unstructured text to structured terms in a table-structured database.

6. A computer system for estimating a labor cost to reconcile semantic conflicts between data schema terms used in different data sources, the computer system comprising:

a CPU, a computer readable memory and a computer readable storage media;

first program instructions to estimate a labor cost for mapping, to shared ontology terms, respective pairs of the data schema terms having semantic conflicts with each other, the first program instructions estimating the labor cost based on at least five of the following;

(a) a number of the data sources that contain the data schema terms having the semantic conflicts, (b) an approximate number of the data schema terms in each of the data sources, (c) an approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) an approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) an approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) an approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) an approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms; and

second program instructions to initiate display on a monitor the estimated labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other; and

wherein

the first and second program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.

7. The computer system of claim 6 wherein the first program instructions estimate the labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other, based on at least six of the following:

(a) the number of the data sources that contain the data schema terms having the semantic conflicts, (b) the approximate number of the data schema terms in each of the data sources, (c) the approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) the approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) the approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) the approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) the approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms.

8. The computer system of claim 6 wherein the first program instructions estimate the labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other, based on all of the following:

(a) the number of the data sources that contain the data schema terms having the semantic conflicts, (b) the approximate number of the data schema terms in each of the data sources, (c) the approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) the approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) the approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) the approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) the approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms.

9. The computer system of claim 6 further comprising:

third first program instructions to estimate a labor cost to identify mistakes in the mappings from a plurality of pairs of the data schema terms to a respective plurality of the shared ontology terms, based on differences in human language text associated with the data schema terms in each of the pairs, and determine correct mappings of the data schema terms in each of the pairs to shared ontology terms; and

wherein

the second program instructions initiate display on the monitor of the estimated labor cost to identify and correct mistakes in the mappings; and

the third program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.

10. The computer system of claim 6 further comprising:

third program instructions to estimate a labor cost to create an information wrapper to map terms in unstructured text to structured terms in a table-structured database; and

the second program instructions initiate display on the monitor of the estimated labor cost to create the information wrapper; and

the third program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.

11. A computer program product for estimating a labor cost to reconcile semantic conflicts between data schema terms used in different data sources, the computer program product comprising:

a computer readable storage media;

first program instructions to estimate a labor cost for mapping, to shared ontology terms, respective pairs of the data schema terms having semantic conflicts with each other, the first program instructions estimating the labor cost based on at least five of the following;

(a) a number of the data sources that contain the data schema terms having the semantic conflicts, (b) an approximate number of the data schema terms in each of the data sources, (c) an approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) an approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) an approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) an approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) an approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms; and

second program instructions to initiate display on a monitor the estimated labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other; and

wherein

the first and second program instructions are stored on the computer readable storage media.

12. The computer program product of claim 11 wherein the first program instructions estimate the labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other, based on at least six of the following:

(a) the number of the data sources that contain the data schema terms having the semantic conflicts, (b) the approximate number of the data schema terms in each of the data sources, (c) the approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) the approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) the approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) the approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) the approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms.

13. The computer program product of claim 11 wherein the first program instructions estimate the labor cost for the mapping, to the shared ontology terms, the data schema terms having semantic conflicts with each other, based on all of the following:

(a) the number of the data sources that contain the data schema terms having the semantic conflicts, (b) the approximate number of the data schema terms in each of the data sources, (c) the approximate labor cost for implementing the shared ontology terms for each of the data sources, (d) the approximate labor cost to manually map to the shared ontology terms a percent of the data schema terms in each of the data sources, (e) the approximate labor cost to validate a percent of the mappings from the data schema terms to the shared ontology terms, (f) the approximate labor cost to perform functional computation for a percent of the mappings from the data schema terms to the shared ontology terms, and (g) the approximate labor cost to perform structural heterogeneity semantic mapping between a percent of the data schema terms and the shared ontology terms.

14. The computer program product of claim 11 further comprising:

third first program instructions to estimate a labor cost to identify mistakes in the mappings from a plurality of pairs of the data schema terms to a respective plurality of the shared ontology terms, based on differences in human language text associated with the data schema terms in each of the pairs, and determine correct mappings of the data schema terms in each of the pairs to shared ontology terms; and

wherein

the second program instructions initiate display on the monitor of the estimated labor cost to identify and correct mistakes in the mappings; and

the third program instructions are stored on the computer readable storage media.

15. The computer program product of claim 11 further comprising:

third program instructions to estimate a labor cost to create an information wrapper to map terms in unstructured text to structured terms in a table-structured database; and

the second program instructions initiate display on the monitor of the estimated labor cost to create the information wrapper; and

the third program instructions are stored on the computer readable storage media.