Method and system for adapting synonym resources to specific domains

• US 20040133418A1
• Filed: 02/09/2004
• Published: 07/08/2004
• Est. Priority Date: 09/29/2000
• Status: Active Grant

• Current Assignee: Matrikon Inc. (Honeywell International Inc.)
• Original Assignee: Gavagai Technology Incorporated
• Inventors: Turcato, Davide, Tisher, Gordon W, Popowich, Frederick P, Nicholson, James Devlan, Fass, Daniel C, Toole, Janine T
• Granted Patent: US 7,330,811 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

A method and system for processing synonyms that adapts a general-purpose synonym resource to a specific domain. The method selects out a domain-specific subset of synonyms from the set of general-purpose synonyms. The synonym processing method in turn comprises two methods that can be used either together or on their own. A method of synonym pruning eliminates those synonyms that are inappropriate in a specific domain. A method of synonym optimization eliminates those synonyms that are unlikely to be used in a specific domain. The method has many applications including, but not limited to, information retrieval and domain-specific thesauri as a writer'"'"'s aid.

Claims

1. A method of adapting a linguistic resource to a specific knowledge domain, wherein said linguistic resource comprises a plurality of target terms each having one or more meanings, each meaning of each target term having associated therewith a set of synonymous terms, said method comprising the steps of:

a) ranking said synonymous terms according to the appropriateness of said synonymous term to said domain; and

b) removing synonymous terms from said linguistic resource according to said ranking.

2. The method of claim 1 wherein said ranking step comprises:

a) forming a synonymy relation between each meaning of said target term and each synonymous term in the set associated with said meaning of said target term;

b) ranking said synonymy relations according to the frequency of occurrence of the target term and associated synonymous term of each synonymy relation in a corpus of data in said domain.

3. The method of claim 2 wherein said ranking step comprises:

a) automatically ranking said synonymy relation according to the frequency of occurrence of said target term and associated synonymous term of each synonymy relation in a corpus of data in said domain;

4. The method of claim 2 wherein said ranking step comprises:

a) automatically ranking said synonymy relation according to the frequency of occurrence of said target term and associated synonymous term of each synonymy relation in a corpus of data in said domain;

b) evaluating the appropriateness of said synonymy relation to said domain by human evaluators acting on the ranking produced by said automatic ranking.

5. The method of claim 2 wherein said ranking step comprises producing a numerical value associated with each synonymy relation representing the appropriateness of said synonymy relation to said domain, where said numerical value is produced from the frequency of occurrence of said terms in each said synonymy relation in a corpus of data in said domain.

6. The method of claim 5 wherein said corpus of data comprises an inventory of previous queries and a searchable corpus of data.

7. The method of claim 2 wherein synonymous terms from said linguistic resource are removed by automatically removing synonymous terms which are ranked below a pre-determined threshold value.

8. The method of claim 2 wherein synonymous terms from said linguistic resource are removed by automatically removing synonymous terms which are ranked below a produced threshold value.

9. The method of claim 2 wherein said method is carried out on a computer system and synonymous terms from said linguistic resource are removed by removing synonymous terms which are ranked below a threshold value set by users through a user interface with the system.

10. The method of claim 2 wherein said linguistic resource is used for information retrieval or as a writers'"'"' aid.

11. The method of claim 1 wherein said ranking step comprises:

a) generating a numerical value representing the appropriateness of said synonymy relations to said domain, where said numerical values are produced by determining the frequency of occurrence of the terms in each said synonymy relation in a corpus of data in said domain, and the frequency of occurrence of words which are semantically related to the target term in said synonymy relation in a corpus of data in said domain.

12. The method of claim 11 wherein said semantically related words are selected from the sets of synonymous terms associated with said target term, the set of words contained in dictionary definitions of said target term, and superordinate and subordinate terms for said target term.

21. The method of claims 1 or 13 wherein said linguistic resource is used for information retrieval.

22. The method of claim 1 or 13 wherein said linguistic resource is used as a writers'"'"' aids.

25. The method of claims 1 or 13 wherein said linguistic resource is a machine-readable dictionary or a machine-readable thesaurus.

27. The method of claim 25 wherein synonymous terms are removed from said linguistic resource by setting a threshold ranking and automatically removing synonymous terms by comparing said ranking to said threshold value.

29. The method of claims 1 or 13 carried out at least in part on a computer.

13. A method of adapting a linguistic resource to a specific knowledge domain, wherein said linguistic resource comprises a plurality of target terms each having one or more meanings, each meaning of each target term having associated therewith a set of synonymous terms, said method comprising minimizing the number of synonymous terms by removing those synonymous terms which are not useful in said domain.

14. The method of claim 13 wherein said method comprises removing synonymous terms which are irrelevant to said domain.

15. The method of claim 13 wherein said method comprises removing sets of synonyms which are redundant in said domain.

16. The method of claim 14 wherein said synonymous terms are determined to be irrelevant by determining the frequency of occurrence of said synonymous terms for a corpus of data in said domain and removing said synonymous terms if said frequency is equal to or less than a pre-determined threshold value.

17. The method of claim 14 wherein said synonymous terms are determined to be irrelevant by determining the frequency of occurrence of said synonymous terms for a corpus of data in said domain and removing said synonymous terms if said frequency is equal to or less than a produced threshold value.

18. The method of claim 14 wherein said synonymous terms are determined to be irrelevant by determining the frequency of occurrence of said synonymous terms for a corpus of data in said domain and removing said synonymous terms if said frequency is equal to or less than a threshold value set by users through a user interface with the system.

19. The method of claim 16 wherein said pre-determined threshold value is set at 0.

20. The method of claims 17, 18 or 19 wherein said pre-determined threshold is variable depending on the size of the domain.

23. The method of claim 15 wherein sets of synonyms which contain a single term which is the same as the target term are considered redundant.

24. The method of claim 15 wherein redundant sets of synonyms are removed by identifying sets of synonyms which are identical to each other and removing all but one of said identical sets.

26. A method of adapting a linguistic resource to a specific knowledge domain, wherein said linguistic resource comprises a plurality of target terms each having one or more meanings, each meaning of each target term having associated therewith a set of synonymous terms, said method comprising:

a) forming a synonymy relation between each meaning of said target term and each synonymous term;

b) automatically ranking said synonymy relation according to the frequency of occurrence of said terms of said synonymy relation in a corpus of data in said domain;

c) evaluating the appropriateness of said synonymy relation to said domain by human evaluators acting on the ranking produced by said automatic ranking;

wherein said automatic ranking step comprises i) producing a numerical value associated with each synonymy relation representing the appropriateness of said synonymy relation to said domain, where said numerical value is produced from the frequency of occurrence of said terms in each said synonymy relation in a corpus of data in said domain and the frequency of occurrence of words which are semantically related to said terms; and

d) removing synonymous terms from said linguistic resource according to said ranking.

28. The method of claim 26 said method is carried out on a computer system and said threshold value is set by users through a user interface with the system.

30. A computer program product for adapting a linguistic resource to a specific knowledge domain, wherein said linguistic resource comprises a plurality of target terms each having one or more meanings, each meaning of each target term having associated therewith a set of synonymous terms, said computer program product comprising:

a computer usable medium having computer readable program code means embodied in said medium for

a) forming a synonymy relation between each meaning of said target term and each synonymous term;

b) automatically ranking said synonymy relation according to the frequency of occurrence of said synonymy relation in a corpus of data in said domain.

c) interfacing with human evaluators to evaluate the appropriateness of said synonymy relation to said domain;

wherein said automatic ranking comprises i) producing a numerical value associated with each synonymy relation representing the appropriateness of said synonymy relation to said domain, where said numerical value is produced from the frequency of occurrence of said terms in each said synonymy relation in a corpus of data in said domain and the frequency of occurrence of words which are semantically related to said terms; and

d) removing synonymous terms from said linguistic resource according to said ranking.

31. A method of adapting a linguistic resource to a specific knowledge domain, wherein said linguistic resource comprises a plurality of target terms each having one or more meanings, each meaning of each target term having associated therewith a set of synonymous terms, said method comprising the steps of:

a) ranking said synonymous terms according to the appropriateness of said synonymous term to said domain; and

b) forming a new linguistic resource which is reduced in size by removing synonymous terms from said linguistic resource according to said ranking.

32. A method of adapting a linguistic resource to a specific knowledge domain, wherein said linguistic resource comprises a plurality of target terms each having one or more meanings, each meaning of each target term having associated therewith a set of synonymous terms, said method comprising forming a new linguistic resource which has a minimum number of synonymous terms by removing those synonymous terms which are not useful in said domain.