Integration of structured data with free text for data mining

• US 20040167908A1
• Filed: 12/05/2003
• Published: 08/26/2004
• Est. Priority Date: 12/06/2002
• Status: Abandoned Application

• Current Assignee: Attensity Corporation (Attensity Group Incorporated)
• Original Assignee: Attensity Corporation (Attensity Group Incorporated)
• Inventors: Bean, David L., Wakefield, Todd D.
• Application Number:
  US10/729,414
• Publication Number:
  US 20040167908A1
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing

Abstract

Disclosed herein are systems, methods and products for interpreting and structuring free text records utilizing extractions of several types including syntactic, role, thematic and domain extractions. Also disclosed herein are systems, methods and products for integrating interpretive extractions with structured data into unified structures that can be analyzed with, among other tools, data mining and data visualization tools.

Claims

1. A computer program product located to one or more storage media devices usable to perform integration of mixed format data, said computer program product comprising instructions executable by a computer to perform the functions of:

accessing a database of structured data, the structured data comprising a set of data tuples;

accessing a source of unstructured data, the unstructured data including free text relatable to the data tuples of the structured data;

interpreting the free text to produce a set of construed data reflecting at least one relational fact conveyed in the free text, each construed datum relatable to a data tuple of the structured data;

integrating the produced data with the data tuples of the structured data, said integrating producing integrated data; and

data mining the integrated data.

2. A computer program product according to claim 1, wherein said accessing a source of unstructured data accesses unstructured data contained within the database of structured data.

3. A computer program product according to claim 1, wherein said accessing a source of unstructured data and said accessing a database of structured data access two separate data sources.

4. A computer program product according to claim 1, wherein said instructions are further executable to perform the function of applying caseframes while performing said interpreting the free text.

5. A computer program product according to claim 1, wherein said instructions are further executable to perform the function of producing a new database containing the integrated data produced by said integrating.

6. A computer program product according to claim 1, wherein said instructions are further executable to perform the function of inserting the produced data into the database of structured data while performing said integrating the produced data.

7. A computer program product according to claim 1, wherein said instructions are further executable to perform the function of creating a new database while performing said integrating the produced data.

8. A computer program product according to claim 7, wherein the instructions are further executable to produce a new relational database containing the integrated data produced by said integrating.

9. A computer program product according to claim 7, wherein the instructions are further executable to produce a file containing the integrated data produced by said integrating.

10. A computer program product according to claim 9, wherein the instructions are further executable to produce a file having a format selected from the group of XML, character separated values, spreadsheet formats and file-based database structures.

11. A computer system including a computer program product according to claim 1, further comprising:

a processing unit coupled to said one or more storage media devices, said processing unit being capable of executing said instructions; and

an execution command unit, whereby operation of said instructions and said processing unit may be commanded or controlled.

12. A computer program product according to claim 1, wherein said instructions are further executable to produce an integrated database while performing said integrating the produced data.

13. A computer program product according to claim 1, wherein the integrated data produced by the performance of said integrating the produced data includes reference information to the original free text for construed data.

14. A computer program product located to one or more storage media devices usable to perform integration of mixed format data, said computer program product comprising instructions executable by a computer to perform the functions of:

accessing a database of structured data, the structured data comprising a set of data tuples;

accessing a source of unstructured data, the unstructured data including free text relatable to the data tuples of the structured data;

interpreting the free text to produce a set of construed data reflecting at least one relational fact conveyed in the free text, each construed datum relatable to a data tuple of the structured data;

integrating the produced data with the data tuples of the structured data, said integrating producing integrated data;

providing the integrated data to a data mining application.

15. A computer program product according to claim 14, wherein said accessing a source of unstructured data accesses unstructured data contained within the database of structured data.

16. A computer program product according to claim 14, wherein said accessing a source of unstructured data and said accessing a database of structured data access two separate data sources.

17. A computer program product according to claim 14, wherein said instructions are further executable to perform the function of applying caseframes while performing said interpreting the free text.

18. A computer program product according to claim 14, wherein said instructions are further executable to perform the function of producing a new database containing the integrated data produced by said integrating.

19. A computer program product according to claim 15, wherein said instructions are further executable to perform the function of inserting the produced data into the database of structured data while performing said integrating the produced data.

20. A computer program product according to claim 14, wherein said instructions are further executable to perform the function of creating a new database while performing said integrating the produced data.

21. A computer program product according to claim 20, wherein the instructions are further executable to produce a new relational database containing the integrated data produced by said integrating.

22. A computer program product according to claim 20, wherein the instructions are further executable to produce a file containing the integrated data produced by said integrating.

23. A computer program product according to claim 22, wherein the instructions are further executable to produce a file having a format selected from the group of XML, character separated values, spreadsheet formats and file-based database structures.

24. A computer system including a computer program product according to claim 14, further comprising:

a processing unit coupled to said one or more storage media devices, said processing unit being capable of executing said instructions; and

an execution command unit, whereby operation of said instructions and said processing unit may be commanded or controlled.

25. A computer program product according to claim 14, wherein said instructions are further executable to store an integrated database while performing said integrating the produced data.

26. A computer program product according to claim 14, wherein the integrated data produced by the performance of said integrating the produced data includes reference information to the original free text for construed data.

27. A method for integrating mixed format data, comprising the steps of:

accessing a database of structured data, the structured data comprising a set of data tuples;

accessing a source of unstructured data, the unstructured data including free text relatable to the data tuples of the structured data;

interpreting the free text to produce a set of construed data reflecting at least one relational fact conveyed in the free text, each construed datum relatable to a data tuple of the structured data;

integrating the produced data with the data tuples of the structured data; and

data mining the integrated data.

28. A method according to claim 27, wherein said accessing a source of unstructured data accesses unstructured data contained within the database of structured data.

29. A method according to claim 27, wherein said accessing a source of unstructured data and said accessing a database of structured data access two separate data sources.

30. A method according to claim 27, wherein said performing said interpreting the free text applies caseframes.

31. A method according to claim 27, further comprising the step of producing a new database containing the integrated data produced by said integrating.

32. A method according to claim 27, further comprising the step of inserting the produced data into the database of structured data.

33. A method according to claim 27, further comprising the step of creating a new database.

34. A method according to claim 33, wherein the new database is a relational database.

35. A method according to claim 33, wherein new database includes at least one file containing the integrated data produced by said integrating.

36. A method according to claim 35, wherein the new database has a format selected from the group of XML, character separated values, spreadsheet formats and file-based database structures.

37. A method according to claim 27, wherein said step of integrating the produced data produces an integrated database.

38. A method according to claim 27, wherein the integrated data produced by the performance of said integrating the produced data includes reference information to the original free text for construed data.