Method and apparatus for automatically structuring free form hetergeneous data

• US 8,996,587 B2
• Filed: 02/15/2007
• Issued: 03/31/2015
• Est. Priority Date: 02/15/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kar, Gautam, Mahindru, Ruchi, Sailer, Anca, Wei, Xing
• Primary Examiner(s): ROBINSON, GRETA LEE
• Application Number:
  US11/675,392
• Publication Number:
  US 20080201279A1
• Time in Patent Office: 2,966 Days
• Field of Search: 705/1, 705/7, 705/1.1, 707/101, 707/102, 707/104.1, 707/5, 707/100, 707/791, 707/796, 707/802, 707/803, 707/811, 707/999.101, 707/602, 707/603, 707/747, 714/46, 714/57, 714/699, 714/701, 706/12, 706/45, 706/13, 704/9, 704/4, 704/5, 704/6, 715/230, 715/231, 715/234
• US Class Current: 707/802
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

Techniques are provided for automatically structuring free form heterogeneous data. In one aspect of the invention, the techniques include obtaining free form heterogeneous data, segmenting the free form heterogeneous data into one or more units, automatically labeling the one or more units based on one or more machine learning techniques, wherein each unit is associated with a label indicating an information type, and structuring the one or more labeled units in a format to facilitate one or more operations that use at least a portion of the labeled units, e.g., information technology (IT) operations.

Claims

1. A method, performed on a data processing system comprising a memory and a data processor coupled to the memory, of automatically structuring free form heterogeneous data, the method comprising the steps of:

obtaining free form heterogeneous data;

segmenting the free form heterogeneous data into one or more units, wherein the one or more units includes a sentence;

automatically labeling the one or more units based on one or more machine learning techniques, wherein each unit is associated with a label indicating an information structure type, wherein automatically labeling one or more units includes labeling a sentence with a label that indicates a type of information provided by the sentence; and

structuring the one or more labeled units in a format to facilitate one or more operations that use at least a portion of the labeled units.

2. The method of claim 1, wherein the step of segmenting the free form heterogeneous data into one or more units comprises identifying punctuation in the free form data.

3. The method of claim 1, wherein the step of segmenting the free form heterogeneous data into one or more units comprises identifying one or more line breaks in the free form data.

4. The method of claim 1, wherein the step of segmenting the free foam heterogeneous data into one or more units comprises identifying a part of speech in the free form data.

5. The method of claim 4, wherein the part of speech is a verb.

6. The method of claim 1, wherein the step of structuring the one or more labeled units in a format comprises structuring the one or more labeled units in a table.

7. The method of claim 1, wherein the step of structuring the one or more labeled units in a format comprises structuring the one or more labeled units in an extensible markup language (XML) format.

8. The method of claim 1, wherein the one or more information technology operations comprises processing at least one of existing free form data and newly obtained free form data.

9. An apparatus for automatically structuring free form heterogeneous data, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

obtain free form heterogeneous data;

segment the free form heterogeneous data into one or more units, wherein the one or more units includes a sentence;

automatically label the one or more units based on one or more machine learning techniques, wherein each unit is associated with a label indicating an information structure type, wherein a sentence is automatically labeled with a label that indicates a type of information provided by the sentence; and

structure the one or more labeled units in a format to facilitate one or more operations that use at least a portion of the labeled units.

10. The apparatus of claim 9, wherein the at least one processor is operative to segment the free form heterogeneous data into one or more units by identifying at least one of punctuation, one or more line breaks, and a part of speech in the free form data.

11. The apparatus of claim 9, wherein the at least one processor is operative to structure the one or more labeled units in a format to facilitate processing of at least one of existing free form data and newly obtained free form data.

12. A computer program product comprising a computer useable storage medium having computer useable program code for automatically structuring free form heterogeneous data, the computer program product including:

computer useable program code for obtaining free form heterogeneous data;

computer useable program code for segmenting the free form heterogeneous data into one or more units, wherein the one or more units includes a sentence;

computer useable program code for automatically labeling the one or more units based on one or more machine learning techniques, wherein each unit is associated with a label indicating an information structure type, wherein automatically labeling one or more units includes labeling a sentence with a label that indicates a type of information provided by the sentence; and

computer useable program code for structuring the one or more labeled units in a format to facilitate one or more operations that use at least a portion of the labeled units.

13. The computer program product of claim 12, wherein the computer usable program code for segmenting the free form heterogeneous data into one or more units comprises computer usable program code for identifying at least one of punctuation, one or more line breaks, and a part of speech in the free form data.

14. The computer program product of claim 12, wherein the computer usable program code for structuring the one or more labeled units in a format comprises computer usable program code for structuring the one or more labeled units in a format to facilitate processing of at least one of existing free foam data and newly obtained free form data.