Systems and methods for lexicon generation

• US 8,527,513 B2
• Filed: 08/26/2010
• Issued: 09/03/2013
• Est. Priority Date: 08/26/2010
• Status: Active Grant

• Current Assignee: RELX Inc. (RELX PLC)
• Original Assignee: LexisNexis Group Inc. (RELX PLC)
• Inventors: Zhang, Paul, Silver, Harry
• Primary Examiner(s): Breene, John E
• Assistant Examiner(s): MYINT, DENNIS Y
• Application Number:
  US12/869,400
• Publication Number:
  US 20120054220A1
• Time in Patent Office: 1,104 Days
• Field of Search: 707/736, 707/758
• US Class Current: 707/736
• CPC Class Codes: G06F 16/374 Thesaurus
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms

Abstract

Disclosed herein are embodiments for lexicon generation. More specifically, at least one embodiment of a method includes determining a corpus term from a plurality of documents, generating a candidate term from the corpus term, and selecting a normalized term from the candidate term and the corpus term. Some embodiments include linking the normalized term with the candidate term and providing an electronic search capability for locating a first document, where the electronic search capability receives the candidate term as a search term and utilizes the normalized term to locate the first document.

Claims

1. A method for lexicon generation, comprising the steps of:

determining a corpus term from a plurality of documents;

generating a candidate term from the corpus term, wherein generating the candidate term comprises generating a linguistic variant of the corpus term;

generating a plurality of equivalent terms from the candidate term;

validating the plurality of equivalent terms by comparing the plurality of equivalent terms to frequency of occurrence of the candidate term;

linking each of the plurality of equivalent terms to the candidate term to create respective equivalent term pairs;

determining whether any of the equivalent term pairs are equivalent and, in response to determining that at least two of equivalent term pairs are equivalent, merging the equivalent term pairs to create a group of equivalent terms;

selecting a normalized term from the group of equivalent terms; and

storing the group of equivalent terms.

2. The method of claim 1, further comprising the steps of:

determining the frequency of occurrence for the candidate term in the first document,

determining whether the frequency of occurrence for the candidate term does not meet a predetermined threshold; and

in response to determining that the frequency of occurrence for the candidate term does not meet the predetermined threshold, removing the candidate term.

3. The method of claim 1, wherein the normalized term is selected based on a frequency of the candidate term and the corpus term in the plurality of documents.

4. The method of claim 1, wherein the electronic search capability includes an electronic searching system for locating legal documents.

5. A system for lexicon generation, comprising:

a processor; and

a memory component that stores lexicon generation logic that when executed by the processor, causes a computer to perform at least the following;

determine a corpus term from a plurality of documents;

generate a candidate term from the corpus term, wherein generating the candidate term comprises generating a linguistic variant of the corpus term;

generate a plurality of equivalent terms from the candidate term;

validate the plurality of equivalent terms by comparing the plurality of equivalent terms to a frequency of occurrence of the candidate term;

link each of the plurality of equivalent terms to the candidate term to create respective equivalent term pairs;

determine whether any of the equivalent term pairs are equivalent and, in response to determining that at least two of equivalent term pairs are equivalent, merging the equivalent term pairs to create a group of equivalent terms;

select a normalized term from the group of equivalent terms; and

store the group of equivalent terms.

6. The system of claim 5, the lexicon generation logic further configured to cause the computer to determine the frequency of occurrence for the candidate term in at least a portion of the plurality of documents.

7. The system of claim 6, the lexicon generation logic further configured to cause the computer to perform the following:

determine whether the frequency of occurrence for the candidate term meets a predetermined threshold; and

in response to determining that the frequency of occurrence for the candidate term meets the predetermined threshold, remove the candidate term.

8. The system of claim 5, wherein the normalized term is selected based on a frequency of the candidate term and the corpus term in the plurality of documents.

9. The system of claim 5, wherein the memory component further stores search logic that is configured to provide an electronic searching system for locating legal documents.

10. A non-transitory computer-readable medium for lexicon generation that stores a program that, when executed by a computer, causes the computer to perform at least the following:

determine a corpus term from a plurality of documents;

generate a candidate term from the corpus term;

term, wherein generating the candidate term comprises generating a linguistic variant of the corpus term;

generate a plurality of equivalent terms from the candidate term;

validate the plurality of equivalent terms by comparing the plurality of equivalent terms to a frequency of occurrence of the candidate term;

link each of the plurality of equivalent terms to the candidate term to create respective equivalent term pairs;

determine whether any of the equivalent term pairs are equivalent and, in response to determining that at least two of equivalent term pairs are equivalent, merging the equivalent term pairs to create a group of equivalent terms;

select a normalized term from the group of equivalent terms; and

store the group of equivalent terms.

11. The non-transitory computer-readable medium of claim 10, the program further causing the computer to determine the frequency of occurrence for the candidate term in at least a portion of the plurality of documents.

12. The non-transitory computer-readable medium of claim 11, the program further causing the computer to perform at least the following:

determine whether the frequency of occurrence for the candidate term meets a predetermined threshold; and

in response to determining that the frequency of occurrence for the candidate term meets the predetermined threshold, remove the candidate term.

13. The non-transitory computer-readable medium of claim 10, wherein the normalized term is selected based on a frequency of the candidate term and the corpus term in the plurality of documents.

14. The non-transitory computer-readable medium of claim 10, the program being further configured to provide an electronic searching system for locating legal documents.

15. The non-transitory computer-readable medium of claim 10, the program further causing the computer to generate an equivalent term to the candidate term.