IDENTIFYING GLOSSARY TERMS FROM NATURAL LANGUAGE TEXT DOCUMENTS

• US 20140163966A1
• Filed: 11/27/2013
• Published: 06/12/2014
• Est. Priority Date: 12/06/2012
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Ramnani, Roshni R., Sengupta, Shubhashis, DWARAKANATH, Anurag, Aggarwal, Aniya
• Granted Patent: US 9,460,078 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/279 Recognition of textual entities

Abstract

A device may obtain text to be analyzed to identify glossary terms. The device may analyze a linguistic unit to generate multiple linguistic units related to the linguistic unit. The device may analyze the multiple linguistic units to generate potential glossary terms. The device may perform a glossary term analysis on the potential glossary terms to generate glossary terms that include a subset of the potential glossary terms. The device may identify included terms that are included in the glossary terms. The device may identify excluded terms that are excluded from the glossary terms. The device may determine a semantic relatedness score between at least one excluded term and at least one included term. The device may selectively add the excluded linguistic term to the glossary terms to form a final set of glossary terms based on the semantic relatedness score, and may output the final set of glossary terms.

Claims

1. A device, comprising:

one or more processors to;

obtain text of a document to be analyzed to identify glossary terms included in the text;

perform a linguistic unit analysis on a linguistic unit, included in the text, to generate a plurality of ambiguous linguistic units from the linguistic unit;

resolve the plurality of ambiguous linguistic units to generate a set of potential glossary terms that includes a subset of the plurality of ambiguous linguistic units;

perform a glossary term analysis on the set of potential glossary terms to generate a set of glossary terms that includes a subset of the set of potential glossary terms;

identify a set of included terms, of the set of potential glossary terms, that are included in the set of glossary terms;

identify a set of excluded terms, of the set of potential glossary terms, that are excluded from the set of glossary terms;

determine a semantic relatedness score between at least one excluded term, of the set of excluded terms, and at least one included term, of the set of included terms;

selectively add the excluded linguistic term to the set of glossary terms to form a final set of glossary terms based on the semantic relatedness score; and

output the final set of glossary terms for the document.

2. The device of claim 1, where the one or more processors, when performing the linguistic unit analysis on the linguistic unit to generate the plurality of ambiguous linguistic units, are further to:

perform at least one of;

a coordinating conjunction analysis that generates the plurality of ambiguous linguistic units from the linguistic unit when the linguistic unit includes a coordinating conjunction,

an adjectival modifier analysis that generates the plurality of ambiguous linguistic units from the linguistic unit when the linguistic unit includes an adjective, or

a headword analysis that generates the plurality of ambiguous linguistic units from the linguistic unit when the linguistic unit includes an abstract noun.

3. The device of claim 1, where the one or more processors, when performing the linguistic unit analysis on the linguistic unit to generate the plurality of ambiguous linguistic units, are further to:

determine that the linguistic unit includes a coordinating conjunction; and

perform a coordinating conjunction analysis that generates the plurality of ambiguous linguistic units from the linguistic unit based on determining that the linguistic unit includes a coordinating conjunction.

4. The device of claim 1, where the one or more processors, when performing the linguistic unit analysis on the linguistic unit to generate the plurality of ambiguous linguistic units, are further to:

determine that the linguistic unit includes an adjective; and

perform an adjectival modifier analysis that generates the plurality of ambiguous linguistic units from the linguistic unit based on determining that the linguistic unit includes an adjective.

5. The device of claim 1, where the one or more processors, when performing the glossary term analysis on the set of potential glossary terms to generate the set of glossary terms, are further to:

perform at least one of;

a multi-word analysis that adds multi-word potential glossary terms, that occur at least a threshold quantity of times in the text, to the set of glossary terms,

a single-word analysis that adds single word nouns, that represent physical entities, to the set of glossary terms,

a verb analysis that adds verbs with a polysemy count of one to the set of glossary terms, or

a process noun analysis that adds process nouns with a polysemy count of one to the set of glossary terms.

6. The device of claim 1, where the linguistic unit includes at least one of:

a multi-word noun phrase,

a multi-word verb phrase,

a single-word noun, or

a single-word verb.

7. A computer-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by one or more processors, cause the one or more processors to;

obtain text to be analyzed to identify glossary terms included in the text;

perform a linguistic unit analysis on a linguistic unit, included in the text, to generate a plurality of linguistic units related to the linguistic unit;

analyze the plurality of linguistic units to generate a set of potential glossary terms that includes a subset of the plurality of linguistic units;

perform a glossary term analysis on the set of potential glossary terms to generate a set of glossary terms that includes a subset of the set of potential glossary terms;

identify a set of included terms, of the set of potential glossary terms, that are included in the set of glossary terms;

identify a set of excluded terms, of the set of potential glossary terms, that are excluded from the set of glossary terms;

determine a semantic relatedness score between at least one excluded term, of the set of excluded terms, and at least one included term, of the set of included terms;

selectively add the excluded linguistic term to the set of glossary terms to form a final set of glossary terms based on the semantic relatedness score; and

output the final set of glossary terms.

8. The computer-readable medium of claim 7, where the one or more instructions, that cause the one or more processors to perform the linguistic unit analysis on the linguistic unit to generate the plurality of linguistic units, further cause the one or more processors to:

perform at least one of;

a coordinating conjunction analysis that generates the plurality of linguistic units from the linguistic unit when the linguistic unit includes a coordinating conjunction,

an adjectival modifier analysis that generates the plurality of linguistic units from the linguistic unit when the linguistic unit includes an adjective, or

a headword analysis that generates the plurality of linguistic units from the linguistic unit when the linguistic unit includes an abstract noun.

9. The computer-readable medium of claim 7, where the one or more instructions, that cause the one or more processors to perform the linguistic unit analysis on the linguistic unit to generate the plurality of linguistic units, further cause the one or more processors to:

determine that the linguistic unit includes an adjective; and

perform an adjectival modifier analysis that generates the plurality of linguistic units from the linguistic unit based on determining that the linguistic unit includes an adjective.

10. The computer-readable medium of claim 7, where the one or more instructions, that cause the one or more processors to perform the linguistic unit analysis on the linguistic unit to generate the plurality of linguistic units, further cause the one or more processors to:

determine that the linguistic unit includes an abstract noun; and

perform a headword analysis that generates the plurality of linguistic units from the linguistic unit based on determining that the linguistic unit includes an abstract noun.

11. The computer-readable medium of claim 7, where the one or more instructions, that cause the one or more processors to perform the glossary term analysis on the set of potential glossary terms to generate the set of glossary terms, further cause the one or more processors to:

perform at least one of;

a multi-word analysis that adds multi-word potential glossary terms, that occur at least a threshold quantity of times in the text, to the set of glossary terms,

a single-word analysis that adds single word nouns, that represent physical entities, to the set of glossary terms,

a verb analysis that adds verbs with a polysemy count of one to the set of glossary terms, or

a process noun analysis that adds process nouns with a polysemy count of one to the set of glossary terms.

12. The computer-readable medium of claim 7, where the one or more instructions, that cause the one or more processors to analyze the plurality of linguistic units to generate the set of potential glossary terms, further cause the one or more processors to:

determine a most frequent linguistic unit, of the plurality of linguistic units, that occurs in the text a greater quantity of times than other linguistic units included in the plurality of linguistic units; and

add the most frequent linguistic unit to the set of potential glossary terms.

13. The computer-readable medium of claim 7, where the one or more instructions, that cause the one or more processors to analyze the plurality of linguistic units to generate the set of potential glossary terms, further cause the one or more processors to:

determine that two or more linguistic units, of the plurality of linguistic units, occur in the text an equal quantity of times, and that no other linguistic units, included in the plurality of linguistic units, occur in the text more than the equal quantity of times;

apply a rule to generate the set of potential glossary terms based on the determination,

the rule including adding at least one of the two or more linguistic units to the set of potential glossary terms.

14. A method, comprising:

obtaining, by a device, text to be analyzed to identify glossary terms included in the text;

performing, by the device, a linguistic unit analysis on a linguistic unit, included in the text, to generate a plurality of ambiguous linguistic units from the linguistic unit;

analyzing, by the device, the plurality of ambiguous linguistic units to generate a set of potential glossary terms that includes a subset of the plurality of ambiguous linguistic units;

performing, by the device, a glossary term analysis on the set of potential glossary terms to generate a set of glossary terms that includes a subset of the set of potential glossary terms;

identifying, by the device, a set of included terms, of the set of potential glossary terms, that are included in the set of glossary terms;

identifying, by the device, a set of excluded terms, of the set of potential glossary terms, that are excluded from the set of glossary terms;

determining, by the device, a semantic relatedness score between an excluded term, of the set of excluded terms, and an included term, of the set of included terms;

selectively adding, by the device, the excluded linguistic term to the set of glossary terms to form a final set of glossary terms based on the semantic relatedness score; and

outputting, by the device, the final set of glossary terms.

15. The method of claim 14, where performing the linguistic unit analysis on the linguistic unit to generate the plurality of ambiguous linguistic units further comprises:

performing at least one of;

a coordinating conjunction analysis that generates the plurality of ambiguous linguistic units from the linguistic unit when the linguistic unit includes a coordinating conjunction,

an adjectival modifier analysis that generates the plurality of ambiguous linguistic units from the linguistic unit when the linguistic unit includes an adjective, or

a headword analysis that generates the plurality of ambiguous linguistic units from the linguistic unit when the linguistic unit includes an abstract noun.

16. The method of claim 14, where performing the linguistic unit analysis on the linguistic unit to generate the plurality of ambiguous linguistic units further comprises:

determining that the linguistic unit includes a coordinating conjunction; and

performing a coordinating conjunction analysis that generates the plurality of ambiguous linguistic units from the linguistic unit based on determining that the linguistic unit includes a coordinating conjunction.

17. The method of claim 14, where performing the linguistic unit analysis on the linguistic unit to generate the plurality of linguistic units further comprises:

determining that the linguistic unit includes an abstract noun; and

performing a headword analysis that generates the plurality of linguistic units from the linguistic unit based on determining that the linguistic unit includes an abstract noun.

18. The method of claim 14, where performing the glossary term analysis on the set of potential glossary terms to generate the set of glossary terms further comprises:

performing at least one of;

a multi-word analysis that adds multi-word potential glossary terms, that occur at least a threshold quantity of times in the text, to the set of glossary terms,

a single-word analysis that adds single word nouns, that represent physical entities, to the set of glossary terms,

a verb analysis that adds verbs with a polysemy count of one to the set of glossary terms, or

a process noun analysis that adds process nouns with a polysemy count of one to the set of glossary terms.

19. The method of claim 14, where selectively adding the excluded linguistic term to the set of glossary terms based on the semantic relatedness score further comprises:

comparing the semantic relatedness score to a threshold value;

performing a first action or a second action based on comparing the semantic relatedness score to the threshold value,

the first action including adding the excluded linguistic term to the set of glossary terms when the semantic relatedness score satisfies the threshold value, and

the second action including excluding the excluded linguistic term from the set of glossary terms when the semantic relatedness score does not satisfy the threshold value.

20. The method of claim 19, where the threshold value is based on a polysemy count of the excluded term.