JOINT RANKING MODEL FOR MULTILINGUAL WEB SEARCH

• US 20100082511A1
• Filed: 09/30/2008
• Published: 04/01/2010
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Niu, Cheng, Hon, Hsiao-Wuen, Zhou, Ming
• Granted Patent: US 8,326,785 B2
• Time in Patent Office: Days
• US Class Current: 706/21
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning

Abstract

Described is a technology in which a classifier is built to rank documents of different languages found in a query based at least in part on similarity to other documents and the relevance of those other documents to the query. A joint ranking model, e.g., based upon a Boltzmann machine, is used to represent the content similarity among documents, and to help determine joint relevance probability for a set of documents. The relevant documents of one language are thus leveraged to improve the relevance estimation for documents of different languages. In one aspect, a hidden layer of units (neurons) represents clusters (corresponding to relevant topics) among the retrieved documents, with an output layer representing the relevant documents and their features, and edges representing a relationship between clusters and documents.

Claims

1. In a computing environment, a method comprising, determining similarity between a first document of first language and a second document of a second, different language, and using the similarity in ranking relevance of the second document with respect to a query submitted in the first language.

2. The method of claim 1 wherein determining the similarity comprises clustering documents of the first language and the second language based upon a query and the query'"'"'s retrieved documents.

3. The method of claim 1 wherein determining the similarity comprises evaluating multilingual features between documents.

4. The method of claim 3 wherein evaluating the multilingual features comprises comparing keywords of each document.

5. The method of claim 1 further comprising, training a classifier to use the similarity in ranking the relevance.

6. The method of claim 5 wherein training the classifier comprises learning a ranking function to estimate comparable scores of documents of different languages.

7. The method of claim 5 wherein the classifier comprises a Boltzmann machine, and wherein training the classifier comprises selecting clusters for the machine, and building the Boltzmann machine with each document and each cluster corresponding to one unit of the Boltzmann machine, with each document unit corresponding to document features, and with edges between each document unit and each cluster unit representing a correlation between that document'"'"'s relevancy and that cluster'"'"'s relevancy.

8. The method of claim 7 wherein training the Boltzmann machine further comprises adjusting weights and thresholds such that a predicted document relevancy distribution approximates a target distribution.

9. The method of claim 7 wherein training the Boltzmann machine further comprises optimizing based upon mean average precision.

10. In a computing environment, a system comprising, a classifier that ranks multilingual documents relative to one another, including means for relating features of documents to topic-based clusters of documents, the classifier ranking documents of different languages relative to a query based upon similarity data between documents according to features of those documents.

11. The system of claim 10 wherein the features of the documents are based upon relevance scores between the query and the document using different fields of the document, inverse document frequencies of query words, query words found in different fields of the document, phrases found in different fields matching with the query, or term frequency of query terms occurring in different fields, or any combination of relevance scores between the query and the document using different fields of the document, inverse document frequencies of query words, query words found in different fields of the document, phrases found in different fields matching with the query, or term frequency of query terms occurring in different fields

12. The system of claim 10 wherein the means for relating features of documents to topic-based clusters of documents comprises a Boltzmann machine, including a first layer containing output units corresponding to document units, and a second layer containing hidden units corresponding to cluster units, in which each document unit is associated with features, and in which edges between the document units and the cluster units correspond to a dependency between document relevancy and cluster relevancy.

13. The system of claim 12 wherein the edges between the document units and a cluster unit comprise edge features, including monolingual similarity between the document and documents in the same language in that cluster using different document fields, cross-lingual similarity between the document and documents in different languages in that cluster using different fields, or similarity between the document and documents in that cluster independent of languages, or any combination of monolingual similarity between the document and the documents in the same language in that cluster using different document fields, cross-lingual similarity between the document and the documents in different languages in that cluster using different fields, or similarity between the document and the documents in that cluster independent of languages.

14. The system of claim 12 further comprising means for training the Boltzmann machine, including means for adjusting weights and thresholds such that a predicted document relevancy distribution approximates a target distribution.

15. The system of claim 12 further comprising means for training the Boltzmann machine, including means for optimizing based upon mean average precision.

16. One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

featurizing a set of documents of different languages into multilingual features;

clustering the set of documents into topic-based clustering data;

constructing a Boltzmann machine based on the documents, the features of the documents, and the clustering data; and

using the Boltzmann machine to rank documents of different languages returned in a search.

17. The one or more computer-readable media of claim 16 having further computer-executable instructions comprising, selecting only a subset of the clustering data for use in the Boltzmann machine based upon cluster relevance.

18. The one or more computer-readable media of claim 16 wherein constructing the Boltzmann machine comprises having each document represented as a document unit corresponding to that document'"'"'s features, having each cluster represented as a cluster unit, and having edges between each document unit and each cluster unit represent a correlation between that document'"'"'s relevancy and that cluster'"'"'s relevancy.

19. The one or more computer-readable media of claim 16 having further computer-executable instructions comprising, training the Boltzmann machine, including adjusting weights and thresholds such that a predicted document relevancy distribution approximates a target distribution.

20. The one or more computer-readable media of claim 16 having further computer-executable instructions comprising, training the Boltzmann machine, including optimizing based upon mean average precision.