Document-based synonym generation

• US 8,762,370 B1
• Filed: 02/08/2013
• Issued: 06/24/2014
• Est. Priority Date: 02/07/2007
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Grushetskyy, Oleksandr, Baker, Steven D.
• Primary Examiner(s): Vital, Pierre
• Assistant Examiner(s): Ly, Anh
• Application Number:
  US13/763,471
• Time in Patent Office: 501 Days
• Field of Search: 707/727, 707/756, 707/804, 707/811, 707/755, 707/730, 707/750, 707/728, 707/E17.072, 707/E17.008, 707/E17.064, 704/4, 704/9, 704/19, 704/205, 704/231, 715/258, 715/260
• US Class Current: 707/727
• CPC Class Codes: G06F 16/9532 Query formulation
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms

Abstract

One embodiment of the present invention provides a system that automatically generates synonyms for words from documents. During operation, this system determines co-occurrence frequencies for pairs of words in the documents. The system also determines closeness scores for pairs of words in the documents, wherein a closeness score indicates whether a pair of words are located so close to each other that the words are likely to occur in the same sentence or phrase. Finally, the system determines whether pairs of words are synonyms based on the determined co-occurrence frequencies and the determined closeness scores. While making this determination, the system can additionally consider correlations between words in a title or an anchor of a document and words in the document as well as word-form scores for pairs of words in the documents.

Claims

1. A computer-implemented method comprising:

selecting a pair of words;

receiving a document having an associated title or anchor;

determining that a first word of the pair of words occurs in the title or anchor of the document and that a different, second word of the pair of words occurs within the document;

determining, by one or more computers, that the first word and the different, second word are synonyms based at least upon determining that the first word of the pair of words occurs in the title or anchor of the document and that the different, second word of the pair of words occurs within the document; and

generating an alternative search query for a search query that includes the first word or the different, second word using the first word as a substitute for the second word in the alternative search query or using the second word as a substitute for the first word in the alternative search query.

2. The method of claim 1, wherein the anchor is a Uniform Resource Locator (URL) anchor.

3. The method of claim 1, comprising:

receiving the search query;

obtaining search results responsive to the alternative search query; and

providing the search results in response to receiving the search query.

4. The method of claim 1, comprising:

determining a co-occurrence frequency for the first word and the second word in a collection of documents; and

determining that the pair of words are synonyms based at least on the determined co-occurrence frequency.

5. The method of claim 1, comprising generating a correlation between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document,

wherein determining that the first word and the different, second word are synonyms based at least upon the generated correlation between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document.

6. The method of claim 1, comprising determining a relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document,

wherein determining that the first word and the different, second word are synonyms is based at least upon the determined relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document.

7. The method of claim 6, wherein determining a relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document comprises determining a frequency that the different, second word of the pair of words occurs in documents when the first word of the pair of words occurs in the title or anchor of the documents.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

selecting a pair of words;

receiving a document having an associated title or anchor;

determining that a first word of the pair of words occurs in the title or anchor of the document and that a different, second word of the pair of words occurs within the document;

determining that the first word and the different, second word are synonyms based at least upon determining that the first word of the pair of words occurs in the title or anchor of the document and that the different, second word of the pair of words occurs within the document; and

generating an alternative search query for a search query that includes the first word or the different, second word using the first word as a substitute for the second word in the alternative search query or using the second word as a substitute for the first word in the alternative search query.

9. The system of claim 8, wherein the anchor is a Uniform Resource Locator (URL) anchor.

10. The system of claim 8, wherein the operations comprises:

receiving the search query;

obtaining search results responsive to the alternative search query; and

providing the search results in response to receiving the search query.

11. The system of claim 8, wherein the operations comprise:

determining a co-occurrence frequency for the first word and the second word in a collection of documents; and

determining that the pair of words are synonyms based at least on the determined co-occurrence frequency.

12. The system of claim 8, wherein the operations comprise generating a correlation between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document,

wherein determining that the first word and the different, second word are synonyms based at least upon the generated correlation between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document.

13. The system of claim 8, wherein the operations comprise determining a relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document,

wherein determining that the first word and the different, second word are synonyms is based at least upon the determined relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document.

14. The system of claim 13, wherein determining a relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document comprises determining a frequency that the different, second word of the pair of words occurs in documents when the first word of the pair of words occurs in the title or anchor of the documents.

15. A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

selecting a pair of words;

receiving a document having an associated title or anchor;

determining that a first word of the pair of words occurs in the title or anchor of the document and that a different, second word of the pair of words occurs within the document;

determining that the first word and the different, second word are synonyms based at least upon determining that the first word of the pair of words occurs in the title or anchor of the document and that the different, second word of the pair of words occurs within the document; and

generating an alternative search query for a search query that includes the first word or the different, second word using the first word as a substitute for the second word in the alternative search query or using the second word as a substitute for the first word in the alternative search query.

16. The computer program product of claim 15, wherein the anchor is a Uniform Resource Locator (URL) anchor.

17. The computer program product of claim 15, wherein the operations comprises:

receiving the search query;

obtaining search results responsive to the alternative search query; and

providing the search results in response to receiving the search query.

18. The computer program product of claim 15, wherein the operations comprise:

determining a co-occurrence frequency for the first word and the second word in a collection of documents; and

determining that the pair of words are synonyms based at least on the determined co-occurrence frequency.

19. The computer program product of claim 15, wherein the operations comprise generating a correlation between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document,

wherein determining that the first word and the different, second word are synonyms based at least upon the generated correlation between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document.

20. The computer program product of claim 15, wherein the operations comprise determining a relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document,

wherein determining that the first word and the different, second word are synonyms is based at least upon the determined relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document.

21. The computer program product of claim 20, wherein determining a relationship between the first word of the pair of words occurring in the title or anchor of the document and the different, second word of the pair of words occurring within the document comprises determining a frequency that the different, second word of the pair of words occurs in documents when the first word of the pair of words occurs in the title or anchor of the documents.