Query selection and results merging

• US 9,805,120 B2
• Filed: 02/14/2014
• Issued: 10/31/2017
• Est. Priority Date: 02/14/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Tawfik, Ahmed Yassin, Kamel, Ahmed Sobhi Mohamed, Lo, Kenneth Chi-Heng
• Primary Examiner(s): Gortayo, Dangelino
• Application Number:
  US14/180,982
• Publication Number:
  US 20150234920A1
• Time in Patent Office: 1,355 Days
• Field of Search: 707748, 707722, 707723, 707724, 707729, 707760
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

Multilingual/cross-region search result merging is described. In various examples, a user enters a query to an information retrieval system and receives search results appropriate to different combinations of human language and geographical region, and where the combinations are selected according to features describing intent of the query. In examples, query intent features are computed using a query understanding component (e.g. a trained query intent classifier) and used, optionally with other query features, to identify combinations of human language and geographical region from a stored profile. In examples the stored profile is computed using data about documents available to an information retrieval system and data about past search behavior. In examples, different combinations of language and region are used to retrieve search results which are merged. For example, a query is translated into another language while preserving calculated intent features of the query.

Claims

1. A computer-implemented method comprising:

upon receiving a query, extracting features comprising language of the query, region of a user, both together called original language-region pair, and intent of the query, from the query;

categorizing the intent of the query based on context of the query;

receiving search results for the query for the original language-region pair;

determining whether the query would benefit from issuing the query to another language-region pair based at least in part on the categorized intent and the features extracted from the query; and

either;

in response to determining that the query would benefit from being issued to another language-region pair;

selecting another language-region pair;

issuing the query to the other language-region pair;

receiving search results for the other language-region pair;

merging the search results for the original language-region pair with search results for the other language-region pair; and

sending to end user equipment the merged search results;

or

in response to determining that the query would not benefit from being issued to another language-region pair, sending to end user equipment the search results for the query for the original language-region pair.

2. The method of claim 1, wherein determining whether the query would benefit from being issued to another language-region pair is performed concurrently with or subsequently to receiving search results for the query for the original language-region pair.

3. The method of claim 1, wherein receiving search results for the query for the original language-region pair comprises issuing the query to a ranker for the original language-region pair.

4. The method of claim 1, wherein receiving search results for the other language-region pair further comprises issuing the query to a ranker for the other language-region pair.

5. The method of claim 4, wherein the ranker for the original language-region pair is trained on the original language-region pair and the ranker for the other language-region pair is trained on the other language-region pair.

6. The method of claim 4, wherein a global ranker is used that takes location and language as inputs and ranks search results accordingly.

7. The method of claim 1, further comprising, after selecting another language-region pair, formulating a new query in the language of the other language-region pair, if the language of the other language-region pair differs from the language of the original language-region pair.

8. The method of claim 7, wherein formulating a new query in the language of the other language-region pair comprises translating the query into the other language of the other language-region pair while preserving the categorized intent of the query or projecting the query to a semantic space and finding the nearest corresponding query from the other language.

9. The method of claim 1, wherein determining whether the query would benefit from being issued to another language-region pair comprises analyzing signals to determine whether the probability of a user wanting the result from a given language-region pair is high.

10. The method of claim 9, wherein the signals are click through rates of the user.

11. The method of claim 1, wherein the categorized intent indicates for a given language-region pair and a given categorized intent at least a further language-region pair providing relevant content.

12. The method of claim 1, wherein determining whether the query would benefit from being issued to another language-region pair depends at least in part on whether the language of the query is a content poor language regarding the intent of the query.

13. The method of claim 1, wherein the extracted features are query and/or user related features.

14. The method of claim 13, wherein the extracted features further comprise at least one of spelling of the query and linguistic profile of the user.

15. The method of claim 1, wherein the language of the query and intent of the query are determined by query understanding techniques, while the user region is determined by a reverse IP lookup or GPS localization techniques.

16. The method of claim 1, wherein merging the search results for the original language-region pair with the search results for the other language-region pair comprises using a result merger which merges the results based on a learned probabilistic model that estimates a relevance gain from inserting a ranked result.

17. The method of claim 1, wherein presenting the merged search results to the user comprises in response to presenting a search result in a language different to the language of the original query, offering the user the possibility of translating the search result into the original language of the query.

18. One or more computer storage media comprising computer-executable instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform a computer-implemented method comprising:

upon receiving a query, extracting features comprising language of the query, region of a user, both together called original language-region pair, and intent of the query, from the query;

categorizing the intent of the query based on context of the query;

receiving search results for the query for the original language-region pair;

determining whether the query would benefit from being issued to another language-region pair based on the categorized intent and the features extracted from the query; and

either;

in response to determining that the query would benefit from being issued to another language-region pair;

selecting another language-region pair;

sending the query to the selected language-region pair;

receiving search results for the selected language-region pair;

merging the search results for the original language-region pair with search results for the selected language-region pair; and

sending to end user equipment the merged search results;

or

in response to determining that the query would not benefit from being issued to another language-region pair, sending to end user equipment the search results for the query for the original language-region pair.

19. The one or more computer storage media of claim 18, wherein receiving search results for the query for the original language-region pair comprises issuing the query to a ranker for the original language-region pair.

20. An information retrieval system, comprising:

at least one hardware processor arranged to control;

a query understanding component arranged to determine intent of a query;

a region identifier arranged to determine a region of a user asking the query;

a language classifier arranged to determine a language of the query;

an intent classifier arranged to determine the intent of the query;

a categorizer arranged to categorize the intent of the query based on context of the query;

a categorizer based content profile that allows to determine whether the query would benefit from being issued to another language-region pair;

a first ranking algorithm component arranged to rank search results according to the region of the user and language of the query;

a second ranking algorithm component arranged to rank search results according to another language-region pair; and

a merging component arranged to merge the search results according to the region of the user and the language of the query with the search results according to the other language-region pair.