Method and Apparatus for Identifying Synonyms and Using Synonyms to Search

• US 20110047138A1
• Filed: 04/23/2010
• Published: 02/24/2011
• Est. Priority Date: 04/27/2009
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Zhang, Qin, Hou, Lei, Guo, Ning, Xing, Fei, Dong, Jing
• Granted Patent: US 8,392,438 B2
• Time in Patent Office: Days
• US Class Current: 707/706
• CPC Class Codes: G06F 16/3338 Query expansion
• CPC Class Codes: G06F 16/374 Thesaurus
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms

Abstract

A method and an apparatus for identifying synonym and utilizing such synonym to conduct search is disclosed. The disclosed method includes: obtaining arbitrary two words to be identified; determining whether a shortest edit distance between the two words less than or equal to an edit distance threshold; determining whether the two words to be identified exist in a preset knowledge database, and if an answer is yes then searching a smallest granularity type with highest weight value for each word in the knowledge database; and if the two word have the same smallest granularity type with highest weight value, then determining such two words are synonyms, or non-synonym otherwise. The disclosed techniques greatly improve accuracy of synonym identification and guarantee effect of synonym identification.

Claims

1. A method for identifying synonyms, the method comprising:

obtaining, by a server, any two words to be identified;

determining that a shortest edit distance between the two words is less than or equal to an edit distance threshold;

determining whether both of the two words exist in a preset knowledge database;

if both of the two words exist in the preset knowledge database, then finding a smallest granularity type with highest weight value for each word in the knowledge database;

if the two words have the same smallest granularity type with highest weight value, then determining that the two words are synonyms; and

if the two words do not have the same smallest granularity type with highest weight value, then determining that the two words are non-synonyms.

2. A method as recited in claim 1, wherein the determining whether both of the two words exist in the preset knowledge database comprises:

segmenting, by the server, one or more unfound words, if any, and then determining whether all of the words after segmentation exist in the knowledge database; and

if all of the words after segmentation exist in the knowledge database, then searching the smallest granularity type with highest weight value for each of such word in the knowledge database.

3. A method as recited in claim 1, wherein the determining two words have the same smallest granularity type with highest weight value comprises:

determining, by the server, whether any changeable character or word of the two words is among changeable characters of a preset common character table;

if any changeable character or word of the two words is among changeable characters of the preset common character table, then determining that the two words are synonyms; and

if any changeable character or word of the two words is not among changeable characters of the preset common character table, then determining that the two words are non-synonyms.

4. A method as recited in claim 1, wherein the knowledge database comprises:

one or more terms and concepts, each term or concept corresponding to at least one type, each type corresponding to the term or concept having a respective weight value.

5. A method as recited in claim 1, wherein the finding a smallest granularity type with highest weight value for each word in the knowledge database comprises:

searching a term or concept corresponding to each word in the knowledge database; and

finding the smallest granularity type with highest weight value for each word according to the at least one type corresponding to the term or concept and relevant weight value.

6. A method as recited in claim 1, further comprising:

if determining that the two words are synonyms, then saving such identified synonyms in a synonym database.

7. A method as recited in claim 6, further comprising:

receiving, by a search engine, a query request from a user, the query request including a query term to be searched;

searching, by the search engine, the query term in the synonym database to find a synonym of the query term;

conducting, by the search engine, a search by using the query term and the synonym of the query term; and

returning, by the search engine, a result including both the query term and the synonym of the query term to the user.

8. An apparatus for identifying synonyms, the apparatus comprising:

a retrieval unit, that obtains any two words to be identified;

a first determination unit, that determines that a shortest edit distance between the two words is less than or equal to an edit distance threshold to inform a second determination unit;

the second determination unit, that determines that both of the two words exist in a preset knowledge database and to inform an query unit;

the query unit, that finds a smallest granularity type with highest weight value for each word in the knowledge database;

a third determination unit, that determines that the two words are synonyms when the two words have the same smallest granularity type with highest weight value, and determines that the two words are non-synonyms when the two words do not have the same smallest granularity type with highest weight value.

9. An apparatus as recited in claim 8, further comprising:

a segmentation unit, that segments one or more unfound words in the knowledge database and then informs the second determination unit, wherein the second determination unit is further configured to determine that all of the words after segmentation exist in the knowledge database and to inform the query unit, and to determine that not all of the words after segment exist in the knowledge database and to inform the segmentation unit.

10. An apparatus as recited in claim 8, further comprising:

a common character table query unit, that determines that changeable words or characters of the words, if any, are among changeable characters of a preset common character table to inform the third determination unit that the two words are synonyms, and determines that changeable words or characters of the words, if any, are not among changeable characters of the preset common character table to inform the third determination unit that the two words are non-synonyms.

11. An apparatus as recited in claim 8, wherein the knowledge database comprises:

one or more terms and concepts, each term or concept corresponding to at least one type, each type corresponding to the term or concept having a weight value.

12. An apparatus as recited in claim 8, wherein the apparatus is a server or a search engine.

13. An apparatus as recited in claim 8, the apparatus further comprising:

a retrieval unit, that receives a query request from a user, the query request including a term to be searched;

a synonym searching unit, that finds a synonym of the term by searching the term in a synonym database;

a search unit, that conducts a search by using the term and the synonym of the term; and

a return unit, that returns a search result to the user.