System and method for hybrid text mining for finding abbreviations and their definitions

• US 7,536,297 B2
• Filed: 01/22/2002
• Issued: 05/19/2009
• Est. Priority Date: 01/22/2002
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Park, Youngja, Byrd, Roy J.
• Primary Examiner(s): Vo; Huyen X.
• Application Number:
  US10/055,279
• Publication Number:
  US 20030139921A1
• Time in Patent Office: 2,674 Days
• Field of Search: 704/260, 704 1- 10, 704251-257, 707/512, 707/6, 707/203, 707/102, 715/512, 715/540
• US Class Current: 704/10
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking

Abstract

This present invention matches one or more abbreviations to one or more definitions. The invention has an abbreviation pattern generation process that generates one or more abbreviation patterns corresponding to the candidate abbreviations, and a definition pattern generation process that generates one or more definition patterns corresponding to the candidate definitions.

Claims

1. A system for matching one or more abbreviations and one or more definitions, comprising:

a recognition process that examines character strings and determines which character strings to be abbreviated;

an abbreviation pattern generation process that creates from said determined character strings one or more abbreviation patterns representing candidate abbreviations, each of the one or more abbreviation patterns being a template that indicates a number and a location of characters and numeric strings within a candidate abbreviation; and

a definition pattern generation process that creates from said determined character strings one or more definition patterns representing candidate definitions, each of the one or more definition patterns being a template that indicates a number and a location of numeric strings, stopwords, prefix/headword combinations and base words within a candidate definition.

2. A system, as in claim 1, further comprising:

a set of abbreviation rules that correlate abbreviation patterns to definition patterns using one or more formation rules that define how each character in an abbreviation is formed from a definition, the set of abbreviation rules including abbreviation rules that have been automatically generated based on candidate abbreviations and definition patterns that have previously been determined as matches by the system;

a lookup process that selects one or more formation rules, being selected formation rules, corresponding to the abbreviation pattern of the candidate abbreviation and the definition pattern of the candidate definition; and

a rule application process that applies the selected formation rules to determine which candidate definitions match the candidate abbreviation.

3. A system, as in claim 2, further comprising:

one or more matching algorithms that match one or more pairs of abbreviations and definitions based on the abbreviation patterns and the definition patterns.

4. A system, as in claim 3, where rule application process and the matching algorithm apply both rule based and non rule based matching processes to match one or more abbreviations and one or more definitions.

5. A system, as in claim 2, where the formation rule that produced the best candidate definition is weighted better due to the choice of the best candidate definition.

6. A system, as in claim 2, further comprising:

a process for adding new abbreviation rules in the existing set of abbreviation rules.

7. A system, as in claim 2, further comprising:

a mechanism for automatically generating one or more new abbreviation rules when no formation rules successfully match high-quality pairs of candidate abbreviations and definitions.

8. A system, as in claim 7, further comprising:

a process for automatically adding the created abbreviation rules to the existing set of abbreviation rules.

9. A system, as in claim 2, further comprising:

a rule generation process for generating abbreviation rules from pairs of abbreviations and definitions.

10. A system, as in claim 1, further comprising:

one or more matching algorithms that match one or more pairs of abbreviations and definitions based on the abbreviation patterns and the definition patterns.

11. A system, as in claim 1, further comprising:

a method for specifying pairs, each of which contains a candidate abbreviation and a candidate definition, for each pair generating an abbreviation patterns and a definition pattern.

12. A system, as in claim 11, where the pairs include one or more of the following:

a user input pair,

an existing abbreviation database pair, and

a pair produced by a program interface.

13. A system, as in claim 1, further comprising:

an abbreviation recognition process that finds one or more candidate abbreviations in text.

14. A system, as in claim 1, further comprising:

a definition finding process that locates one or more candidate definitions corresponding to the candidate abbreviation.

15. A system, as in claim 1, further comprising:

a best match selection process that chooses a best candidate definition from the matched candidate definitions using one or more criteria.

16. A system, as in claim 15, further comprising:

a best match selection mechanism that employs one or more weighting features.

17. A system, as in claim 16, where the weighting features may include one or more of the followings:

syntactic cues found in the context,

rule priority of the formation rule that matched the pair,

the distance of the abbreviation and the definition,

capitalization of the definition,

number of words in the definition, and

number of stopwords in the definition.

18. A system, as in claim 1, further comprising:

an output process that outputs the candidate abbreviation and the matched candidate definition as confirmed pairs.

19. A system, as in claim 1, further comprising:

a set of layered matching algorithms which are based on the relationship between the lengths of abbreviation patterns and the lengths of definition patterns.

20. A system, as in claim 19, where each algorithm in the layered matching mechanism is applied in priority sequence.

21. A system for matching one or more abbreviations and one or more definitions, comprising:

means for examining character strings and determining which character strings to be abbreviated;

means for generating from said determined character strings one or more abbreviation patterns representing candidate abbreviations, each of the one or more abbreviation patterns being a template that indicates a number and a location of characters and numeric strings within a candidate abbreviation; and

means for generating from said determined character strings one or more definition patterns representing candidate definitions, each of the one or more definition patterns being a template that indicates a number and a location of numeric strings, stopwords, prefix/headword combinations and base words within a candidate definition.

22. A method for matching one or more abbreviations and one or more definitions, comprising:

examining character strings and determining which character strings to be abbreviated;

generating from said determined character strings one or more abbreviation patterns representing candidate abbreviations, each of the one or more abbreviation patterns being a template that indicates a number and a location of characters and numeric strings within a candidate abbreviation; and

generating from said determined character strings one or more definition patterns representing candidate definitions, each of the one or more definition patterns being a template that indicates a number and a location of numeric strings, stopwords, prefix/headword combinations and base words within a candidate definition.