Machine Translation for Query Expansion

• US 20080319962A1
• Filed: 03/17/2008
• Published: 12/25/2008
• Est. Priority Date: 06/22/2007
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Riezler, Stefan, Vasserman, Alexander L.
• Granted Patent: US 9,002,869 B2
• Time in Patent Office: Days
• US Class Current: 707/4
• CPC Class Codes: G06F 16/3338 Query expansion

Abstract

Methods, systems and apparatus, including computer program products, for expanding search queries. One method includes receiving a search query, selecting a synonym of a term in the search query based on a context of occurrence of the term in the received search query, the synonym having been derived from statistical machine translation of the term, and expanding the received search query with the synonym and using the expanded search query to search a collection of documents. Alternatively, another method includes receiving a request to search a corpus of documents, the request specifying a search query, using statistical machine translation to translate the specified search query into an expanded search query, the specified search query and the expanded search query being in the same natural language, and in response to the request, using the expanded search query to search a collection of documents.

Claims

1. A computer-implemented method comprising:

receiving a search query;

selecting a synonym of a term in the search query based on a context of occurrence of the term in the received search query, the synonym having been derived from statistical machine translation of the term;

expanding the received search query with the synonym; and

using the expanded search query to search a collection of documents.

2. The method of claim 1, further comprising:

identifying a plurality of recorded search queries, the term occurring in each of the recorded search queries;

using statistical machine translation to translate the plurality of recorded search queries into respective translated search queries;

identifying a plurality of potential synonyms from the translated search queries, a potential synonym being one or more unique translations of the term among the translated search queries, each potential synonym having an associated context of occurrence; and

selecting the synonym from the plurality of potential synonyms by matching the context of occurrence of the term in the received query with the context of occurrence of each potential synonym in the translated search query.

3. The method of claim 1, wherein the statistical machine translation uses bidirectional phrase alignments.

4. The method of claim 1, further comprising:

identifying question phrases and corresponding answer phrases from a plurality of documents; and

building a translation model for the statistical machine translation using the question phrases as a source language and the corresponding answer phrases as a target language.

5. The method of claim 1, further comprising:

identifying a first phrase in a first natural language;

generating a second phrase in a second natural language by translating the first phrase into a second natural language;

identifying a paraphrase of the first phrase by translating the second phrase back into the first natural language; and

building a translation model for the statistical machine translation using the first phrase as a source language and the paraphrase as a corresponding target language.

6. The method of claim 1, further comprising:

identifying a search result access log, each record in the search result access log identifying a respective recorded search query and a corresponding snippet, the snippet of a respective recorded search query being a portion of content from a document accessed by a user, the document having been presented to the user as a search result in response to receiving the respective recorded search query; and

building a translation model for the statistical machine translation using the search queries from the search result access log as a source language and the corresponding snippets as a target language.

7. The method of claim 6, further comprising:

filtering records from the query log based on respective information associated with each record, the respective information being one or more of;

a position of the document with respect to other documents presented as search results to the user;

an amount of time elapsed between having provided the search result to the user and the user accessing the document; and

an amount of time elapsed between the user having accessed the document and the user performing a subsequent operation.

8. The method of claim 6, wherein the portion of content from a document is one or more of:

a title of the document;

anchor terms associated with the document; and

an excerpt of the document, the excerpt containing terms from the respective recorded search query.

9. A computer-implemented method comprising:

receiving a request to search a corpus of documents, the request specifying a search query;

using statistical machine translation to translate the specified search query into an expanded search query, the specified search query and the expanded search query being in the same natural language; and

in response to the request, using the expanded search query to search a collection of documents.

10. A computer program product, encoded on a tangible program carrier, operable to cause data processing apparatus to perform operations comprising:

receiving a search query;

selecting a synonym of a term in the search query based on a context of occurrence of the term in the received search query, the synonym having been derived from statistical machine translation of the term;

expanding the received search query with the synonym; and

using the expanded search query to search a collection of documents.

11. The computer program product of claim 10, further operable to perform operations comprising:

identifying a plurality of recorded search queries, the term occurring in each of the recorded search queries;

using statistical machine translation to translate the plurality of recorded search queries into respective translated search queries;

identifying a plurality of potential synonyms from the translated search queries, a potential synonym being one or more unique translations of the term among the translated search queries, each potential synonym having an associated context of occurrence; and

selecting the synonym from the plurality of potential synonyms by matching the context of occurrence of the term in the received query with the context of occurrence of each potential synonym in the translated search query.

12. The computer program product of claim 10, wherein the statistical machine translation uses bidirectional phrase alignments.

13. The computer program product of claim 10, further operable to perform operations comprising:

identifying question phrases and corresponding answer phrases from a plurality of documents; and

building a translation model for the statistical machine translation using the question phrases as a source language and the corresponding answer phrases as a target language.

14. The computer program product of claim 10, further operable to perform operations comprising:

identifying a first phrase in a first natural language;

generating a second phrase in a second natural language by translating the first phrase into a second natural language;

identifying a paraphrase of the first phrase by translating the second phrase back into the first natural language; and

building a translation model for the statistical machine translation using the first phrase as a source language and the paraphrase as a corresponding target language.

15. The computer program product of claim 10, further operable to perform operations comprising:

identifying a search result access log, each record in the search result access log identifying a respective recorded search query and a corresponding snippet, the snippet of a respective recorded search query being a portion of content from a document accessed by a user, the document having been presented to the user as a search result in response to receiving the respective recorded search query; and

building a translation model for the statistical machine translation using the search queries from the search result access log as a source language and the corresponding snippets as a target language.

16. The computer program product of claim 15, further operable to perform operations comprising:

filtering records from the query log based on respective information associated with each record, the respective information being one or more of;

a position of the document with respect to other documents presented as search results to the user;

an amount of time elapsed between having provided the search result to the user and the user accessing the document; and

an amount of time elapsed between the user having accessed the document and the user performing a subsequent operation.

17. The computer program product of claim 15, wherein the portion of content from a document is one or more of:

a title of the document;

anchor terms associated with the document; and

an excerpt of the document, the excerpt containing terms from the respective recorded search query.

18. A computer program product, encoded on a tangible program carrier, operable to cause data processing apparatus to perform operations comprising:

receiving a request to search a corpus of documents, the request specifying a search query;

using statistical machine translation to translate the specified search query into an expanded search query, the specified search query and the expanded search query being in the same natural language; and

in response to the request, using the expanded search query to search a collection of documents.

19. A system comprising:

one or more computers configured to perform operations including;

receiving a search query;

selecting a synonym of a term in the search query based on a context of occurrence of the term in the received search query, the synonym having been derived from statistical machine translation of the term;

expanding the received search query with the synonym; and

using the expanded search query to search a collection of documents.

20. The system of claim 19, the system further configured to perform operations comprising:

identifying a plurality of recorded search queries, the term occurring in each of the recorded search queries;

using statistical machine translation to translate the plurality of recorded search queries into respective translated search queries;

identifying a plurality of potential synonyms from the translated search queries, a potential synonym being one or more unique translations of the term among the translated search queries, each potential synonym having an associated context of occurrence; and

selecting the synonym from the plurality of potential synonyms by matching the context of occurrence of the term in the received query with the context of occurrence of each potential synonym in the translated search query.

21. The system of claim 19, wherein the statistical machine translation uses bidirectional phrase alignments.

22. The system of claim 19, the system further configured to perform operations comprising:

identifying question phrases and corresponding answer phrases from a plurality of documents; and

building a translation model for the statistical machine translation using the question phrases as a source language and the corresponding answer phrases as a target language.

23. The system of claim 19, the system further configured to perform operations comprising:

identifying a first phrase in a first natural language;

generating a second phrase in a second natural language by translating the first phrase into a second natural language;

identifying a paraphrase of the first phrase by translating the second phrase back into the first natural language; and

building a translation model for the statistical machine translation using the first phrase as a source language and the paraphrase as a corresponding target language.

24. The system of claim 19, the system further configured to perform operations comprising:

identifying a search result access log, each record in the search result access log identifying a respective recorded search query and a corresponding snippet, the snippet of a respective recorded search query being a portion of content from a document accessed by a user, the document having been presented to the user as a search result in response to receiving the respective recorded search query; and

building a translation model for the statistical machine translation using the search queries from the search result access log as a source language and the corresponding snippets as a target language.

25. The system of claim 24, the system further configured to perform operations comprising:

filtering records from the query log based on respective information associated with each record, the respective information being one or more of;

a position of the document with respect to other documents presented as search results to the user;

an amount of time elapsed between having provided the search result to the user and the user accessing the document; and

an amount of time elapsed between the user having accessed the document and the user performing a subsequent operation.

26. The system of claim 24, wherein the portion of content from a document is one or more of:

a title of the document;

anchor terms associated with the document; and

an excerpt of the document, the excerpt containing terms from the respective recorded search query.

27. A system comprising:

one or more computers configured to perform operations including;

receiving a request to search a corpus of documents, the request specifying a search query;

using statistical machine translation to translate the specified search query into an expanded search query, the specified search query and the expanded search query being in the same natural language; and

in response to the request, using the expanded search query to search a collection of documents.