Ranking search results using language types

• US 7,792,833 B2
• Filed: 04/26/2006
• Issued: 09/07/2010
• Est. Priority Date: 03/03/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Meyerzon, Dmitriy, Zaragoza, Hugo
• Primary Examiner(s): Jalil; Neveen Abel
• Assistant Examiner(s): Syed; Farhan M
• Application Number:
  US11/412,723
• Publication Number:
  US 20060294100A1
• Time in Patent Office: 1,595 Days
• Field of Search: None
• US Class Current: 707/730
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

Search results of a search query on a network are ranked according to an additional ranking function for the prior probability of relevance of a document based on document property. The ranking function can be adjusted based on a comparison of the language that a document is written in and the language that is associated with a search query. Both query-independent values and query-dependent values can be used to rank the document.

Claims

1. A computer-implemented method for ranking search results, comprising:

determining a first property associated with each document in a collection of documents;

wherein the first property is a language type associated with the document that identifies a language of the document;

wherein the language of the document is determined by performing a statistical analysis of a character distribution in the document and comparing it to a trained language character distribution;

storing an identified language for each of the documents when it is determined that the identified language is not a default language in a language storage that is a query independent rank (QIR) storage that is separate from a QIR storage that stores other values used at query time;

determining a query language of a search query;

estimating a ranking value corresponding to properties for each document, wherein the ranking value corresponds to a measure of the relevance of each document based on the search query;

ranking each document that is responsive to the search query to obtain the search results, wherein each document is ranked based on the estimated ranking value and a comparison of the query language with the first property value;

ranking the documents according to a scoring function (score) that is determined according to at least;

a computed click distance (CD), a weight of a query-independent component (w_cd), a weight of the click distance (b_cd), a weight of a URL depth (b_ud), the URL depth (UD) and a click distance saturation constant (K_cd); and

using the ranking of the documents to display the search results.

2. The computer-implemented method of claim 1, wherein the comparison lowers the ranking of the document when the query language is different from the document language.

3. The computer-implemented method of claim 1, wherein the comparison lowers the ranking of the document when the query language class is different from the document language class.

4. The computer-implemented method of claim 1, wherein the comparison lowers the ranking of the document when the query language is different from the document language except when the document language is English.

5. The computer-implemented method of claim 1, wherein the comparison lowers the ranking of the document when the query language is different from the document language except, when the document language is English, the comparison lowers the ranking of the document to a lesser extent than if the document language is a different language other than English.

6. The computer-implemented method of claim 1, wherein determining the language of the document is performed during an indexing process of the documents for searching.

7. The computer-implemented method of claim 1, wherein the query language is determined from a language setting that is derived from a browser used to initiate the search query.

8. The computer-implemented method of claim 1, wherein ranking each document further comprises using a component corresponding to the first property in a scoring function for determining a relevance score for each of the documents.

9. The computer-implemented method of claim 1, wherein the first property for a document is stored within a pseudo-key that associates the first property with the document in the index.

10. The computer-implemented method of claim 1, wherein the scoring function (score) is further determined according to the weight of the prior probability of relevance based on a language match (W(l_d, l_q)).

11. The computer-implemented method of claim 10, wherein the scoring function (score) is further determined according to file type priors.

12. The computer-implemented method of claim 11, wherein the scoring function (score) is further determined according to:

a weighted term frequency (wtf), a weighted document length (wdl), an average weighted document length (avwdl), a number of documents on the network (N);

a number of documents containing a query term (n), and other constants (k₁, b).

13. The computer-implemented method of claim 12, wherein the scoring function (score) is given by the following formula;

wherein each value within the scoring function is a definite value, wherein W(t) is a file type component; and

wherein the weight of the prior probability of relevance based on a language match (W(l_d, l_q));

$\sum <br><br>\frac{\mathrm{wtf}<br><br>\left(k_1+1\right)}{k_1<br><br>\left(\left(1-b\right)+b<br><br>\frac{\mathrm{wdl}}{\mathrm{avwdl}}\right)+\mathrm{wtf}}\times <br><br>\log <br><br>\left(\frac{N}{n}\right)+w_{\mathrm{cd}}<br><br>\frac{1}{k_{\mathrm{cd}}+\frac{b_{\mathrm{cd}}<br><br>\mathrm{CD}+b_{\mathrm{ud}}<br><br>\mathrm{UD}}{b_{\mathrm{cd}}+b_{\mathrm{ud}}}}+W<br><br>\left(t\right)+W<br><br>\left(l_d,l_q\right)$

14. A system for ranking search results, comprising:

a processor;

a search engine included on a computing device, the search engine configured to execute computer-executable instructions, the computer-executable instructions comprising;

determining a first property associated with each document in a collection of documents;

wherein the first property is a language type that identifies a language of the document, wherein the language type for each of the documents is only stored in a separate QIR (Query Independent Rank) storage when the language type is not a default language from another QIR storage used for storing values that can be used at query time for searching each document;

determining a query language of a search query;

estimating a ranking value corresponding to properties for each document, wherein the ranking value corresponds to a measure of the relevance of each document based on the search query; and

ranking each document that is responsive to the search query to obtain the search results, wherein each document is ranked based on the estimated ranking value and a comparison of the query language with the first property value;

ranking the documents according to a scoring function (score) that is determined according to at least;

a computed click distance (CD), a weight of a query-independent component (w_cd), a weight of the click distance (b_cd), a weight of a URL depth (b_ud), the URL depth (UD), a click distance saturation constant (K_cd), a weighted term frequency (wtf), a weighted document length (wdl), an average weighted document length (avwdl), a number of documents on the network (N);

a number of documents containing a query term (n); and

using the ranking of the documents to display the search results.

15. The system of claim 14, wherein the scoring function (score) is further determined according to at least:

the first property for a language match (W(l_d, l_q)).

16. The system of claim 15, wherein the scoring function (score) is further determined according to:

other constants (k₁, b), a file type component W(t), the weight of the prior probability of relevance based on a language match (W(l_d, l_q)), and the scoring function (score) is given by;

$\sum <br><br>\frac{\mathrm{wtf}<br><br>\left(k_1+1\right)}{k_1<br><br>\left(\left(1-b\right)+b<br><br>\frac{\mathrm{wdl}}{\mathrm{avwdl}}\right)+\mathrm{wtf}}\times <br><br>\log <br><br>\left(\frac{N}{n}\right)+w_{\mathrm{cd}}<br><br>\frac{1}{k_{\mathrm{cd}}+\frac{b_{\mathrm{cd}}<br><br>\mathrm{CD}+b_{\mathrm{ud}}<br><br>\mathrm{UD}}{b_{\mathrm{cd}}+b_{\mathrm{ud}}}}+W<br><br>\left(t\right)+W<br><br>\left(l_d,l_q\right)$

17. A computer-readable storage medium that includes computer-executable instructions for ranking search results, the computer-executable instructions comprising:

determining a first property associated with each document in a collection of documents;

wherein the first property is a language type associated with the document;

wherein the collection of documents comprises documents of a default language and documents not of a default language;

wherein the language type identifies a language of the document and is only stored in a language QIR (Query Independent Rank) storage when the language type is not the default language;

wherein a separate QIR storage from the language QIR storage is used for storing values that can be used at query time for searching each document;

determining a query language of a search query;

estimating a ranking value corresponding to properties for each document, wherein the ranking value corresponds to a measure of the relevance of each document based on the search query;

ranking each document that is responsive to the search query to obtain the search results, wherein each document is ranked based on the estimated ranking value and a comparison of the query language with the first property value; and

using the ranking of the documents to display the search results.

18. The computer-readable medium of claim 17, wherein the comparison lowers the ranking of the document when the query language is different from the document language.

19. The computer-readable medium of claim 17, wherein the comparison lowers the ranking of the document when the query language class is different from the document language class when the document language is English.

20. The computer-readable medium of claim 17, further comprising adjusting a query-independent value for each document, as additional search queries are performed and a relevance measure of the search results based on the file types is made.