Integration of structured data with relational facts from free text for data mining

• US 20040167887A1
• Filed: 12/05/2003
• Published: 08/26/2004
• Est. Priority Date: 12/06/2002
• Status: Abandoned Application

• Current Assignee: Attensity Corporation (Attensity Group Incorporated)
• Original Assignee: Attensity Corporation (Attensity Group Incorporated)
• Inventors: Bean, David L., Wakefield, Todd D.
• Application Number:
  US10/729,883
• Publication Number:
  US 20040167887A1
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing

Abstract

Disclosed herein are systems, methods and products for interpreting and structuring free text records utilizing extractions of several types including syntactic, role, thematic and domain extractions. Also disclosed herein are systems, methods and products for integrating interpretive extractions with structured data into unified structures that can be analyzed with, among other tools, data mining and data visualization tools.

Claims

1. A computer program product located to one or more storage media devices usable to perform integration of mixed format data, said computer program product comprising instructions executable by a computer to perform the functions of:

accessing a database of structured data, the structured data comprising a set of data tuples;

accessing a source of unstructured data, the unstructured data including free text relatable to the data tuples of the structured data;

extracting relational facts from the free text;

producing a set of construed data reflecting at least one relational fact conveyed in the free text, each construed datum containing at least one relational fact, each construed datum being further relatable to a data tuple of the structured data;

integrating the produced data with the data tuples of the structured data; and

data mining the integrated data.

2. A computer program product according to claim 1, wherein said accessing a source of unstructured data accesses unstructured data contained within the database of structured data.

3. A computer program product according to claim 1, wherein said accessing a source of unstructured data and said accessing a database of structured data access two separate data sources.

4. A computer program product according to claim 1, wherein said instructions are further executable to perform the function of applying caseframes while performing said interpreting the free text.

5. A computer program product according to claim 1, wherein said instructions are further executable to perform the function of producing a new database containing the integrated data produced by said integrating.

6. A computer program product according to claim 1, wherein said instructions are further executable to perform the function of inserting the produced data into the database of structured data while performing said integrating the produced data.

7. A computer program product according to claim 1, wherein said instructions are further executable to perform the function of creating a new database while performing said integrating the produced data.

8. A computer program product according to claim 7, wherein the instructions are further executable to produce a new relational database containing the integrated data produced by said integrating.

9. A computer program product according to claim 8, wherein the instructions are further executable to produce a file containing the integrated data produced by said integrating.

10. A computer program product according to claim 9, wherein the instructions are further executable to produce a file having a format selected from the group of XML, character separated values, spreadsheet formats and file-based database structures.

11. A computer system including a computer program product according to claim 1, further comprising:

a processing unit coupled to said one or more storage media devices, said processing unit being capable of executing said instructions; and

an execution command unit, whereby operation of said instructions and said processing unit may be commanded or controlled.

12. A computer program product according to claim 1, wherein said instructions are further executable to combine like attributes for the extracted relational facts produced in performing said extracting relational facts from the free text.

13. A computer program product according to claim 1, wherein said instructions are further executable to combine like relation types for the extracted relational facts produced in performing said extracting relational facts from the free text.

14. A computer program product according to claim 1, wherein said instructions provide relational facts with domain roles applied in performing said extracting relational facts from the free text.

15. A computer program product according to claim 1, wherein said instructions store the relational facts produced in performing said extracting relational facts from the free text.

16. A computer program product according to claim 1, wherein the extracted relational facts produced in performing said extracting relational facts and the integrated data produced by the performance of said integrating the produced data includes reference information to the original free text.

17. A computer program product located to one or more storage media devices usable to perform integration of mixed format data, said computer program product comprising instructions executable by a computer to perform the functions of:

accessing a database of structured data, the structured data comprising a set of data tuples;

accessing a source of unstructured data, the unstructured data including free text relatable to the data tuples of the structured data;

extracting relational facts from the free text;

producing a set of construed data reflecting at least one relational fact conveyed in the free text, each construed datum containing at least one relationship, each construed datum being further relatable to a data tuple of the structured data;

integrating the produced data with the data tuples of the structured data; and

providing the integrated data to a data mining application.

18. A computer program product according to claim 17, wherein said accessing a source of unstructured data accesses unstructured data contained within the database of structured data.

19. A computer program product according to claim 17, wherein said accessing a source of unstructured data and said accessing a database of structured data access two separate data sources.

20. A computer program product according to claim 17, wherein said instructions are further executable to perform the function of applying caseframes while performing said interpreting the free text.

21. A computer program product according to claim 17, wherein said instructions are further executable to perform the function of producing a new database containing the integrated data produced by said integrating.

22. A computer program product according to claim 17, wherein said instructions are further executable to perform the function of inserting the produced data into the database of structured data while performing said integrating the produced data.

23. A computer program product according to claim 17, wherein said instructions are further executable to perform the function of creating a new database while performing said integrating the produced data.

25. A computer program product according to claim 23, wherein the instructions are further executable to produce a new relational database containing the integrated data produced by said integrating.

26. A computer program product according to claim 25, wherein the instructions are further executable to produce a file containing the integrated data produced by said integrating.

27. A computer program product according to claim 26, wherein the instructions are further executable to produce a file having a format selected from the group of XML, character separated values, spreadsheet formats and file-based database structures.

28. A computer system including a computer program product according to claim 17, further comprising:

a processing unit coupled to said one or more storage media devices, said processing unit being capable of executing said instructions; and

an execution command unit, whereby operation of said instructions and said processing unit may be commanded or controlled.

29. A computer program product according to claim 17, wherein said instructions are further executable to combine like attributes for the extracted relational facts produced in performing said extracting relational facts from the free text.

30. A computer program product according to claim 17, wherein said instructions are further executable to combine like relations for the extracted relational facts produced in performing said extracting relational facts from the free text.

31. A computer program product according to claim 17, wherein said instructions provide relational facts with domain roles applied in performing said extracting relational facts from the free text.

32. A computer program product according to claim 17, wherein said instructions store the relational facts produced in performing said extracting relational facts from the free text.

33. A computer program product according to claim 17, wherein the extracted relational facts produced in performing said extracting relational facts and the integrated data produced by the performance of said integrating the produced data includes reference information to the original free text.

34. A method for integrating mixed format data, comprising the steps of:

accessing a database of structured data, the structured data comprising a set of data tuples;

accessing a source of unstructured data, the unstructured data including free text relatable to the data tuples of the structured data;

extracting relationships from the free text;

producing a set of construed data reflecting at least one relational fact conveyed in the free text, each construed datum containing at least one relational fact, each construed datum being further relatable to a data tuple of the structured data; and

integrating the produced data with the data tuples of the structured data.

35. A method according to claim 34, wherein said accessing a source of unstructured data accesses unstructured data contained within the database of structured data.

36. A method according to claim 34, wherein said accessing a source of unstructured data and said accessing a database of structured data access two separate data sources.

37. A method according to claim 34, wherein said performing said interpreting the free text applies caseframes.

38. A method according to claim 34, further comprising the step of producing a new database containing the integrated data produced by said integrating.

39. A method according to claim 34, further comprising the step of inserting the produced data into the database of structured data.

40. A method according to claim 34, further comprising the step of creating a new database.

41. A method according to claim 40, wherein the new database is a relational database.

42. A method according to claim 34, wherein new database includes at least one file containing the integrated data produced by said integrating.

43. A method according to claim 42, wherein the new database has a format selected from the group of XML, character separated values, spreadsheet formats and file-based database structures.

44. A method according to claim 34, further comprising the step of combining like attributes for the extracted relational facts produced in performing said extracting relational facts from the free text.

45. A method according to claim 34, further comprising the step of combining like relation types for the extracted relational facts produced in performing said extracting relational facts from the free text.

46. A method according to claim 34, wherein domain roles are applied in said step of extracting relational facts from the free text.

47. A method according to claim 34, further comprising the step of storing the relational facts produced in performing said extracting relational facts from the free text.

48. A method according to claim 34, wherein the extracted relational facts produced in performing said extracting relational facts and the integrated data produced by the performance of said integrating the produced data includes reference information to the original free text.