Knowledge discovery tool extraction and integration

• US 8,356,036 B2
• Filed: 02/19/2008
• Issued: 01/15/2013
• Est. Priority Date: 02/04/2005
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Bechtel, Michael E., Mathur, Sanjay, Arago, Jordi
• Primary Examiner(s): Jalil, Neveen Abel
• Assistant Examiner(s): Young, Kevin L
• Application Number:
  US12/070,457
• Publication Number:
  US 20080147590A1
• Time in Patent Office: 1,792 Days
• Field of Search: None
• US Class Current: 707/748
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: G16H 70/00 ICT specially adapted for the handling or processing of medical references

Abstract

A method for integrating a data item into a knowledge model is provided. The method may include retrieving the data item from a data source, determining if the data item has been previously integrated into the knowledge model, and integrating the data element into the knowledge model if the data item has not been previously integrated.

Claims

1. A computer system comprising:

a processor; and

memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to perform operations comprising;

providing, in a database, a plurality of entity table types relevant to a domain type, wherein each entity table type includes a respective entity table name field and at least one respective data item field;

populating the respective entity table name fields and the respective data item fields by automatically;

extracting a plurality of data items from a plurality of data sources;

determining respective confidence values for the extracted data items based on a plurality of pre-determined confidence values, wherein the pre-determined confidence values, wherein a respective predetermined confidence value is provided for each respective entity table name and each respective data item according to which data source of the plurality of data sources provided the respective entity table name and the respective data item; and

integrating the extracted data items into respective entity table name fields and data item fields based on the respective confidence values for the extracted data items;

determining relationship types between at least some data items of the plurality of data items that are integrated into a plurality of tables in the database, wherein the relationship types are based on respective data sources from which the data items have been extracted, the relationship types including at least two of the following;

a first relationship type, a second relationship type, a third relationship type, and a fourth relationship type, wherein;

the first relationship type is between first and second data items extracted from a first structured data source,

the second relationship type is between data items extracted from different structured data sources, wherein the second relationship type is based on a plurality of respective first relationship types,

the third relationship type is between a data item extracted from a second structured data source, and an entity table including, as entity table data items, attributes about a first unstructured data source, and wherein the data item extracted from the second structured data source is mentioned in the first unstructured data source, and the fourth relationship type is between two data items that;

have been integrated from different structured data sources, and are both mentioned in a second unstructured data source; and

storing the relationship types in a computer system memory.

2. The computer system of claim 1 wherein the operations further comprise:

updating the database if a previously integrated data item has been modified in a data source;

determining whether a previously integrated data item has been modified by automatically;

determining a first data item value for the previously integrated data item;

determining a second data item value based on the data item in the data source; and

comparing the first data item value to the second data item value, wherein the first and second data item values are respective hash values based on, respectively, the previously integrated data item and the data item in the data source.

3. The computer system of claim 1 wherein the operations further comprise automatically establishing a field-to-field link in the database, wherein the field-to-field link is selected from the group consisting of:

a field-to-filed link between two respective data items if both respective data items were integrated from a single structured data source;

a field-to-field link between a first entity table name field and a second entity table name field if both the first and second entity table name fields were integrated with data items from a single structured data source; and

a field-to-field link between a first entity table name field of a first entity table and a data item field of a second entity table if both the first entity table name field and the data item field of the second entity table were integrated with data items from a single structured data source.

4. The computer system of claim 1 wherein the operations further comprise establishing a transitive link within the database, wherein the transitive link is selected from the group consisting of:

a one-hop transitive link between a first integrated data item and a second integrated data item if the first integrated data item and a third integrated data item were both integrated from a first structured data source, and the second integrated data item and the third integrated data item were both integrated from a second structured data source; and

a multi-hop transitive link between a first integrated data item and a second integrated data item based on a plurality of data items integrated from a plurality of structured data sources.

5. The computer system of claim 1 wherein the operations further comprise:

establishing in the database an entity table that contains attributes about an unstructured data source, including a title of the unstructured data source, wherein the unstructured data source is one of;

an article, a document, a web page, an HTML file, and a spreadsheet; and

establishing in the database a field-to-text link between an integrated data item from a structured data source and the entity table if the integrated data item is mentioned in the unstructured data source.

6. The computer system of claim 1 wherein the operations further comprise:

establishing an entity table that contains attributes about an unstructured data source, including a title of the unstructured data source; and

establishing a proximity link between two data items integrated from respective structured data sources if the two data items are both mentioned in the unstructured data source, wherein the proximity link is further based upon a proximity definition selected from the group consisting of;

proximity within a paragraph, proximity within a sentence, proximity within a predetermined number of words, and proximity within a spreadsheet.

7. The computer system of claim 1 wherein the operations further comprise assigning confidence levels between links in the database, wherein a first confidence level is assigned to each field-to-field link within the database, a second confidence level is assigned to each field-to-text link within the database, a third confidence level is assigned to each transitive link in the database, and a fourth confidence level is assigned to each proximity link in the database.

8. The computer system of claim 1 wherein the operations further comprise:

determining a synonym for a first extracted data item; and

integrating the first extracted data item into the database based on the synonym, wherein the synonym is integrated into the database.

9. The computer system of claim 1 wherein the operations further comprise:

displaying an active panel data item in an active panel of a multiple panel user display;

displaying in other panels of the multiple panel user display, simultaneously with the display of the active panel data item, entity tables having data items that have a relationship to the active panel data item, wherein respective relationships between the active panel data item and respective data items displayed in the other panels are each one of;

a field-to-field link, a field-to-text link, a transitive link, and a proximity link; and

displaying, on a user interface, relationship type indicia to indicate relationship types between displayed data items.

10. The computer system of claim 1, wherein the pre-determined confidence values indicate reliability of each of the data sources based on at least one of an internal review of the data sources and an external review of the data sources, wherein the internal review is performed by reviewers selected from the group consisting of:

reviewers comprising users of a system implementing the operations, and reviewers comprising developers of a system implementing the operations;

wherein the external review is performed using a third party reliability standard; and

wherein the operations further comprise;

comparing a first confidence value indicating reliability of a third data item stored in a data item field to a second confidence value indicating reliability of a data source having a fourth data item for the data item field;

updating the database by integrating the fourth data item into the data item field if the second confidence value is greater than the first confidence value;

associating time stamps with the extracted data items; and

using the time stamps for the third and fourth data items to determine which of the third and fourth items is more recent if the first and second confidence values are the same, wherein the more recent of the third and fourth data items is integrated into the data item field in the database if the first and second confidence values are the same.

11. A computer system comprising:

a processor; and

memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to perform operations comprising;

providing, in a database, a plurality of entity table types relevant to a domain type, wherein each entity table type includes a respective entity table name field and at least one respective data item field;

populating the respective entity table name fields and the respective data item fields by automatically;

extracting a plurality of data items from a plurality of data sources;

determining respective confidence values for the extracted data items based on a plurality of pre-determined confidence values, wherein a respective predetermined confidence value is provided for each respective entity table name and each respective data item according to which data source of the plurality of data sources provided the respective entity table name and the respective data item;

integrating the extracted data items into respective entity table name fields and data item fields based on the respective confidence values for the extracted data items;

storing a first data item extracted from a first data source in a first data item field and storing a second data item extracted from the first data source in a second data item field;

populating a first row of a direct relationship table having a plurality of rows with a first direct relationship definition indicating that the first data item field and the second data item field store respective data items that have been extracted from the first data source;

searching the direct relationship table for a second row having a second direct relationship definition indicating that a third data item field stores a third data item that has been extracted from a second data source from which the first data item has also been extracted, wherein the second data source is different than the first data source;

determining a transitive relationship definition, based on the first direct and second direct relationship definitions from the direct relationship table, indicating that the second data item field is related to the third data item field, wherein the transitive relationship definition is based on at least two separate relationships between data item fields; and

storing the third relationship definition in a transitive relationship table.

12. The computer system of claim 11 wherein the operations further comprise:

updating the database if a previously integrated data item has been modified in a data source;

determining whether a previously integrated data item has been modified by automatically;

determining a first data item value for the previously integrated data item;

determining a second data item value based on the data item in the data source; and

comparing the first data item value to the second data item value, wherein the first and second data item values are respective hash values based on, respectively, the previously integrated data item and the data item in the data source.

13. The computer system of claim 11 wherein the operations further comprise automatically establishing a field-to-field link in the database, wherein the field-to-field link is selected from the group consisting of:

a field-to-filed link between two respective data items if both respective data items were integrated from a single structured data source;

a field-to-field link between a first entity table name field and a second entity table name field if both the first and second entity table name fields were integrated with data items from a single structured data source; and

a field-to-field link between a first entity table name field of a first entity table and a data item field of a second entity table if both the first entity table name field and the data item field of the second entity table were integrated with data items from a single structured data source.

14. The computer system of claim 11 wherein the operations further comprise establishing a transitive link within the database, wherein the transitive link is selected from the group consisting of:

a one-hop transitive link between a first integrated data item and a second integrated data item if the first integrated data item and a third integrated data item were both integrated from a first structured data source, and the second integrated data item and the third integrated data item were both integrated from a second structured data source; and

a multi-hop transitive link between a first integrated data item and a second integrated data item based on a plurality of data items integrated from a plurality of structured data sources.

15. The computer system of claim 11 wherein the operations further comprise:

establishing in the database an entity table that contains attributes about an unstructured data source, including a title of the unstructured data source, wherein the unstructured data source is one of;

an article, a document, a web page, an HTML file, and a spreadsheet; and

establishing in the database a field-to-text link between an integrated data item from a structured data source and the entity table if the integrated data item is mentioned in the unstructured data source.

16. The computer system of claim 11 wherein the operations further comprise:

establishing an entity table that contains attributes about an unstructured data source, including a title of the unstructured data source; and

establishing a proximity link between two data items integrated from respective structured data sources if the two data items are both mentioned in the unstructured data source, wherein the proximity link is further based upon a proximity definition selected from the group consisting of;

proximity within a paragraph, proximity within a sentence, proximity within a predetermined number of words, and proximity within a spreadsheet.

17. The computer system of claim 11 wherein the operations further comprise assigning confidence levels between links in the database, wherein a first confidence level is assigned to each field-to-field link within the database, a second confidence level is assigned to each field-to-text link within the database, a third confidence level is assigned to each transitive link in the database, and a fourth confidence level is assigned to each proximity link in the database.

18. The computer system of claim 11 wherein the operations further comprise:

determining a synonym for a first extracted data item; and

integrating the first extracted data item into the database based on the synonym, wherein the synonym is integrated into the database.

19. The computer system of claim 11 wherein the operations further comprise:

displaying an active panel data item in an active panel of a multiple panel user display;

displaying in other panels of the multiple panel user display, simultaneously with the display of the active panel data item, entity tables having data items that have a relationship to the active panel data item, wherein respective relationships between the active panel data item and respective data items displayed in the other panels are each one of;

a field-to-field link, a field-to-text link, a transitive link, and a proximity link; and

displaying, on a user interface, relationship type indicia to indicate relationship types between displayed data items.

20. The computer system of claim 11, wherein the pre-determined confidence values indicate reliability of each of the data sources based on at least one of an internal review of the data sources and an external review of the data sources, wherein the internal review is performed by reviewers selected from the group consisting of:

reviewers comprising users of a system implementing the operations, and reviewers comprising developers of a system implementing the operations;

wherein the external review is performed using a third party reliability standard; and

wherein the operations further comprise;

comparing a first confidence value indicating reliability of a third data item stored in a data item field to a second confidence value indicating reliability of a data source having a fourth data item for the data item field;

updating the database by integrating the fourth data item into the data item field if the second confidence value is greater than the first confidence value;

associating time stamps with the extracted data items; and

using the time stamps for the third and fourth data items to determine which of the third and fourth items is more recent if the first and second confidence values are the same, wherein the more recent of the third and fourth data items is integrated into the data item field in the database if the first and second confidence values are the same.