System and method for language extraction and encoding utilizing the parsing of text data in accordance with domain parameters

• US 6,182,029 B1
• Filed: 08/06/1999
• Issued: 01/30/2001
• Est. Priority Date: 10/28/1996
• Status: Expired due to Term

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Friedman, Carol
• Primary Examiner(s): Thomas, Joseph
• Application Number:
  US09/370,329
• Time in Patent Office: 543 Days
• Field of Search: 704/9, 704/1, 704/10, 705/2, 705/3, 707/104, 707/513, 707/531, 707/532, 707/533, 707/534, 706/45, 706/46, 706/55, 706/59
• US Class Current: 704/9
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation

Abstract

A computerized method for extracting information from natural-language text data includes parsing the text data to determine the grammatical structure of the text data and regularizing the parsed text data to form structured word terms. The parsing step, which can be performed in one or more parsing modes, includes the step of referring to a domain parameter having a value indicative of a domain from which the text data originated, wherein the domain parameter corresponds to one or more rules of grammar within a knowledge base related to the domain to be applied for parsing the text data. Preferably, the structured output is mapped back to the words in the original sentences of the text data input using XML tags.

Claims

1. A method for extracting information from natural-language text data, comprising:

parsing the text data to determine the grammatical structure of the text data, said parsing step comprising the step of referring to a domain parameter having a value indicative of a domain from which the text data originated, the domain parameter corresponding to one or more rules of grammar within a knowledge base related to the domain to be applied for parsing the text data;

regularizing the parsed text data to form structured word terms; and

tagging the text data with a structured data component derived from the structured word terms.

2. The method according to claim 1, further comprising preprocessing the data prior to parsing, with preprocessing comprising the step of referring to the domain parameter.

3. The method according to claim 1, further comprising the step of encoding at least one term of the regularized phrase, with encoding comprising the step of referring to the domain parameter.

4. The method according to claim 1, further comprising filtering the regularized phrase.

5. The method according to claim 1, further comprising referring to an additional parameter which is indicative of the degree to which subphrase parsing is to be carried out.

6. The method according to claim 1, further comprising referring to an additional parameter which is indicative of desired filtering.

7. The method according to claim 1, further comprising referring to an additional parameter which is indicative of a desired type of output.

8. The method according to claim 1, further comprising referring to an additional parameter which is indicative of a desired output format.

9. The method according to claim 1, wherein said parsing step further comprises segmenting the text data by sentences.

10. The method according to claim 1, wherein said parsing step further comprises:

segmenting the text data by sentences; and

segmenting each of the sentences at identified words or phrases.

11. The method according to claim 1, wherein said parsing step further comprises:

segmenting the text data by sentences; and

segmenting each of the sentences at a prefix.

12. The method according to claim 1, wherein said parsing step further comprises skipping undefined words.

13. The method according to claim 1, wherein said parsing step further comprises:

identifying one or more primary findings in the text data; and

identifying one or more modifiers associated with the primary findings.

14. The method according to claim 1, further comprising performing error recovery when parsing of the text data is unsuccessful.

15. The method according to claim 14, wherein said error recovery step comprises:

segmenting the text data; and

analyzing the segmented text data to achieve at least a partial parsing of the unsuccessfully parsed text data.

16. The method according to claim 1, wherein said tagging step comprises providing the structured data component in a Standard Generalized Markup Language (SGML) compatible format.

17. The method according to claim 1, wherein said tagging step comprises providing the structured data component in Extensible Markup Language (XML).

18. The method according to claim 1, further comprising highlighting one or more primary findings in the natural-language text data.

19. A computer system for extracting information from natural-language text data, comprising:

means for parsing the natural-language text data, said parsing means comprising means for referring to a domain parameter having a value indicative of a domain from which the natural-language text data originated, and wherein the domain parameter corresponds to one or more rules of grammar within a knowledge base related to the domain to be applied for parsing the natural-language text data;

means for regularizing the parsed text data to form structured word terms; and

means for tagging the text data with a structured data component derived from the structured word terms.

20. The system according to claim 19, further comprising means for preprocessing the data prior to parsing, with the preprocessing means comprising means for referring to the domain parameter.

21. The system according to claim 19, further comprising means for encoding at least one term of the regularized phrase, with the encoding means comprising means for referring to the domain parameter.

22. The system according to claim 19, further comprising means for filtering the regularized phrase.

23. The system according to claim 19, further comprising means for referring to an additional parameter which is indicative of the degree to which subphrase parsing is to be carried out.

24. The system according to claim 19, further comprising means for referring to an additional parameter which is indicative of desired filtering.

25. The system according to claim 19, further comprising means for referring to an additional parameter which is indicative of a desired type of output.

26. The system according to claim 19, further comprising means for referring to an additional parameter which is indicative of a desired output format.

27. The system according to claim 19, wherein said parsing means further comprises means for segmenting the text data by sentences.

28. The system according to claim 19, wherein said parsing means further comprises:

means for segmenting the text data by sentences; and

means for segmenting each of the sentences at identified words or phrases.

29. The system according to claim 19, wherein said parsing means further comprises:

means for segmenting the text data by sentences; and

means for segmenting each of the sentences at a prefix.

30. The system according to claim 19, wherein said parsing means further comprises means for skipping undefined words.

31. The system according to claim 19, wherein said parsing means further comprises:

means for identifying one or more primary findings in the text data; and

means for identifying one or more modifiers associated with the primary findings.

32. The system according to claim 19, further comprising means for performing error recovery when parsing of the text data is unsuccessful.

33. The system according to claim 32. wherein said error recovery means comprises:

means for segmenting the text data; and

means for analyzing the segmented text data to achieve at least a partial parsing of the unsuccessfully parsed text data.

34. The system according to claim 19, wherein said tagging means comprises means for providing the structured data component in a Standard Generalized Markup Language (SGML) compatible format.

35. The system according to claim 19, wherein said tagging step comprises means for providing the structured data component in Extensible Markup Language (XML).

36. The system according to claim 19, further comprising means for highlighting one or more primary findings in the original text data.

37. A combination of the system according to claim 19 with an interface module for enabling the system to receive input from and/or to produce standardized output for the World-Wide Web and/or a local network.

38. The combination according to claim 37, further comprising means for viewing the output using a standardized browser.

39. The combination according to claim 38, wherein the browser is a Web-browser.