Structured term recognition

• US 10,339,214 B2
• Filed: 11/02/2012
• Issued: 07/02/2019
• Est. Priority Date: 11/04/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Glass, Michael R., Gliozzo, Alfio M.
• Primary Examiner(s): Spooner, Lamont M
• Assistant Examiner(s): Ogunbiyi, Oluwadamilola M
• Application Number:
  US13/667,729
• Publication Number:
  US 20130117024A1
• Time in Patent Office: 2,433 Days
• Field of Search: 704257, 704E15019
• CPC Class Codes: G06F 40/237 Lexical tools
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules

Abstract

A method, system and computer program product for recognizing terms in a specified corpus. In one embodiment, the method comprises providing a set of known terms t∈T, each of the known terms t belonging to a set of types Γ (t)={γ₁, . . . }, wherein each of the terms is comprised of a list of words, t=w₁, w₂, . . . , w_n, and the union of all the words for all the terms is a word set W. The method further comprises using the set of terms T and the set of types to determine a set of pattern-to-type mappings p→γ; and using the set of pattern-to-type mappings to recognize terms in the specified corpus and, for each of the recognized terms in the specified corpus, to recognize one or more of the types γ for said each recognized term.

Claims

1. computer implemented method of recognizing types of terms in a specified corpus, comprising:

providing a set of known terms t∈

T, each of the known terms t belonging to a given set of types Γ

(t)={γ

₁, γ

₂. . . }, wherein each of the terms is comprised of a list of words, t=w₁, w₂. . . , w_n, and the union of all the words w for all the terms t is a word set W;

forming, by a clustering component of a computer system, a multitude of clusters of words from the words in W;

using, by a mapping determining component of the computer system, the set of known terms T and the given set of types Γ

to determine a set of pattern-to-type mappings {p₁→

γ

₁, p₂→

γ

₂, . . .}, each of the pattern-to-type mappings p→

γ

mapping an associated sequence of the clusters of words to one or more of the given set of types {γ

₁, γ

₂, . . . };

using, by a term recognition component of the computer system, the determined set of pattern-to-type mappings to recognize corpus terms in the specified corpus, each of the corpus terms being comprised of two or more words; and

for each of the recognized corpus terms in the specified corpus, using, by a word mapping component of the computer system, a specified context in the corpus of at least a plurality of the words of said each corpus term to map said plurality of words to one of the sequences of the clusters of words, and using, by a type recognition component of the computer system, one of the determined pattern-to-type mappings to map said one of the sequences of the clusters of words to one or more of the types y of the given set of types {γ

₁, γ

₂, . . . } to recognize said one or more of the types γ

of the given set of types for said each recognized corpus term to boost performance of term recognition systems based on dictionary lookup while extending coverage of ontologies.

2. The method according to claim 1, wherein the using the set of terms T and the set of types Γ

to determine a set of pattern-to-type mappings includes;

forming a set of rules R, based on said clusters, to map the terms t to the types γ

.

3. The method according to claim 2, wherein at least one of the words of each of the terms is assigned to at least one of the clusters.

4. The method according to claim 3, wherein the forming a set of rules R includes forming one of the rules p→

γ

to map one of the terms t to one of the types, based on the clusters to which the words of said one of the terms are assigned.

5. The method according to claim 2, wherein the rules map patterns of the clusters to the types γ

.

6. The method according to claim 2, wherein each word of each term is assigned to one and only one of the clusters.

7. The method according to claim 2, wherein each pattern of the clusters maps to at most one of the types γ

.

8. The method according to claim 2, wherein the words are assigned to the clusters based on distributional similarity of the words.

9. The method according to claim 1, wherein the using the set of pattern-to-type mappings to recognize terms in the specified corpus includes:

selecting a group of words from the specified corpus;

identifying one or more contexts in the specified corpus for each of the selected group of words; and

grouping said selected group of words based on a defined similarity of the co s of the selected words.

10. The method according to claim 1, wherein:

the using a specified context in the corpus of at least a plurality of the words of each of the corpus terms to map said plurality of words to one of the sequences of the clusters of words includes;

for said at least one of the words of the each corpus term, constructing a context vector for said one word, said context vector for said one word including

a set of adjacent words, wherein each of said adjacent words is present in the specified corpus within a given number of words of at least one occurrence of said one word in the specified corpus, and

for each of the adjacent words, a value representing the number of times said each adjacent word is present in the specified corpus within the given number of words of all occurrences of said one word in the specified corpus; and

using the context vector to select the one of the pattern-to-type mappings.

11. The method according to claim 1, wherein:

the word set W is in a medical dictionary;

the recognized corpus terms in the specified corpus are in a medical domain; and

each of the recognized corpus terms includes a word indicating a part of a human anatomy.

12. A computer system for recognizing types of terms in a specified corpus, the computer comprising:

a memory for storing data;

one or more processing units connected to the memory to transmit data to and to receive data from the memory, and configured for;

receiving a set of known terms t∈

T, each of the known terms t belonging to a given set of types Γ

(t)={γ

₁, γ

₂, . . .}, wherein each of the terms is comprised of a list of words, t=w₁, w₂. . . , w_n, and the union of all the words w for all the terms t is a word set W;

forming, by a clustering component of the computer system a multitude of clusters of words from the words in W;

using, by a mapping determining component of the computer system, the set of known terms T and the given set of types Γ

to determine a set of pattern-to-type mappings {p₁→

γ

₁, p₂→

γ

₂. . . }, each of the pattern-to-type mappings p→

γ

mapping an associated sequence of the clusters of words to one or more of the given set of types {γ

₁, γ

₂, . . .}; and

using, by a term recognition component of the computer system, the determined set of pattern-to-type mappings to recognize corpus terms in the specified corpus, each of the corpus terms being comprised of two or more words; and

for each of the recognized corpus terms in the specified corpus, using, by a word mapping component of the computer system, a specified context in the corpus of at least a plurality of the words of said each corpus term to map said plurality of words to one of the sequences of the clusters of words, and using, by a type recognition component of the computer system, one of the determined pattern-to-type mappings to map said one of the sequences of the clusters of words to one or more types γ

of the given set of types {γ

₁, γ

₂, . . .} to recognize said one or more of the types γ

of the given set of types for said each recognized corpus term to boost performance of term recognition systems based on dictionary lookup while extending coverage of ontologies.

13. The system according to claim 12, wherein the using the set of terms T and the set of types to determine a set of pattern-to-type mappings includes:

forming a set of rules R, based on said clusters, to map the terms t to the types γ

.

14. The system according to claim 13, wherein:

at least one of the words of each of the terms is assigned to at least one of the clusters; and

the forming a set of rules R includes forming one of the rules p to map one of the terms t to one of the types γ

, based on the clusters to which the words of said one of the terms are assigned.

15. The system according to claim 12, wherein the using the set of pattern-to-type mappings to recognize terms in the specified corpus includes:

selecting a group of words from the specified corpus;

identifying one or more contexts in the specified corpus for each of the selected group of words; and

grouping said selected group of words based on a defined similarity of the contexts of the selected words.

16. An article of manufacture comprising:

at least one computer readable hardware medium tangibly embodying a program of instructions for recognizing types of terms in a specified corpus, said program of instructions, when executing in a computer, performing the following;

receiving at the computer a set of known terms t∈

T, each of the known terms t belonging to a given set of types Γ

(t)={γ

₁, γ

₂, . . .}, wherein each of the terms is comprised of a list of words, t=w₁, w₂. . . , w_n, and the union of all the words w for all the terms t is a word set W;

forming at a clustering component of the computer a multitude of clusters of words from the words in W;

using at a mapping determining component of the computer the set of known terms T and the given set of types Γ

to determine a set of pattern-to-type mappings {p₁→

γ

₁, p₂→

γ

₂, . . .}, each of the pattern-to-type mappings p→

γ

mapping as associated sequence of the clusters of words to one or more of the given set of types {γ

₁, γ

₂, . . .}; and

using at a term recognition component of the computer the determined set of known pattern-to-type mappings to recognize corpus terms in the specified corpus, each of the corpus terms being comprised of two or more words; and

for each of the recognized corpus terms in the specified corpus, using at a word mapping component of the computer a specified context in the corpus of at least a plurality of the words of said each corpus term to map said plurality of words to one of the sequences of the clusters of words, and using, at a type recognition component of the computer system, one of the determined pattern-to-type mappings to map said one of the sequences of the clusters of words to one or more types γ

of the given set of types {γ

₁, γ

₂, . . .} to recognize said one or more of the types γ

of the given set of types for said each recognized corpus term to boost performance of term recognition systems based on dictionary lookup while extending coverage of ontologies.

17. The article of manufacture according to claim 16, wherein the using the set of terms T and the set of types to determine a set of pattern-to-type mappings includes:

forming a set of rules R, based on said clusters, to map the terms t to the types γ

.

18. The article of manufacture according to claim 17, wherein:

at least one of the words of each of the terms is assigned to at least one of the clusters; and

the forming a set of rules R includes forming one of the rules p to map one of the terms t to one of the types , based on the clusters to which the words of said one of the terms are assigned.

19. The article of manufacture according to claim 16, wherein the using the set of pattern-to-type mappings to recognize terms in the specified corpus includes:

selecting a group of words from the specified corpus;

identifying one or more contexts in the specified corpus for each of the selected group of words; and

grouping said selected group of words based on a defined similarity of the contexts of the selected words.