Adaptive query suggestion

• US 10,108,699 B2
• Filed: 01/22/2013
• Issued: 10/23/2018
• Est. Priority Date: 01/22/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Song, Ruihua, Chen, Yu, Wen, Ji-Rong
• Primary Examiner(s): Tran, Loc
• Application Number:
  US13/747,274
• Publication Number:
  US 20140207746A1
• Time in Patent Office: 2,100 Days
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence

Abstract

When a user-submitted query is received, a set of candidate queries is identified. For each of the candidate queries, features are extracted that, for each candidate query, reflect a measure of effectiveness of the candidate query. The candidate queries are rank ordered based on the measure of effectiveness, and one or more of the top-ranked candidate queries are presented as suggested alternatives to the user-submitted query.

Claims

1. A method comprising:

receiving a user-submitted query originating from a user; and

in response to receiving the user-submitted query, and prior to returning search results associated with the user-submitted query;

identifying a plurality of candidate queries, without further input from the user;

for each candidate query, without further input from the user;

extracting three or more features, each feature reflecting a measurement of effectiveness of the candidate query with respect to the user-submitted query, wherein the three or more features including an estimated normalized discounted cumulative gain, wherein the measurement of effectiveness of the candidate query based, at least in part, on a match feature that reflects how well the candidate query matches search results of the candidate query, a cross match feature that reflects how well the user-submitted query matches search results of the candidate query, and a similarity feature that reflects similarities between search results of the user-submitted query and search results of the candidate query; and

generating a feature vector that includes each of the three or more features as individual components of the feature vector;

rank ordering the candidate queries based at least in part on the feature vectors of the candidate queries; and

suggesting one or more of the top-ranked candidate queries as alternate queries for the user-submitted query.

2. A method as recited in claim 1, wherein the match feature includes at least one of:

a title match that reflects how well the candidate query matches titles of the search results of the candidate query;

a snippet match that reflects how well the candidate query matches snippets of the search results of the candidate query;

or a URL match that reflects how well the candidate query matches URLs of the search results of the candidate query.

3. A method as recited in claim 1, wherein the cross match feature includes at least one of:

a title cross match that reflects how well the user-submitted query matches titles of the search results of the candidate query;

a snippet cross match that reflects how well the user-submitted query matches snippets of the search results of the candidate query;

or a URL cross match that reflects how well the user-submitted query matches URLs of the search results of the candidate query.

4. A method as recited in claim 1, wherein the similarity feature includes at least one of:

a page similarity that reflects a similarity between the content of the search results of the user-submitted query and the content of the search results of the candidate query;

a URL similarity that reflects a number of common URLs between the search results of the user-submitted query and the search results of the candidate query;

or a domain similarity that reflects a number of common domains between the search results of the user-submitted query and the search results of the candidate query.

5. A method as recited in claim 1, wherein the feature vector includes, in addition to the estimated normalized discounted cumulative gain, a title match, a snippet match, a uniform resource locator (URL) match, a title cross match, a snippet cross match, a URL cross match, a page similarity, an URL similarity, and a domain similarity.

6. A method as recited in claim 1, wherein a value of the estimated normalized discounted cumulative gain is calculated based, at least in part, on an estimated relevance rating.

7. A method as recited in claim 6, wherein the estimated relevance rating represents, for each document in a set of search results, a number of times that the document is returned in the top n results of the candidate queries associated with the user-submitted query.

8. A method as recited in claim 1, wherein rank ordering the candidate queries based at least in part on the feature vectors comprises:

learning a ranking model based on training data; and

applying the ranking model to the feature vectors for each of the candidate queries.

10. One or more non-transitory computer storage media encoded with computer-executable instructions that, when executed, configure a computer system to perform a method as recited in claim 1.

9. A method comprising:

receiving a user-submitted query originating from a user;

determining whether the user-submitted query is a difficult query, wherein a difficult query is a query that will not return relevant or authoritative results;

in an event that the user-submitted query is not a difficult query;

sending the user-submitted query to a search engine;

receiving search results associated with the user-submitted query from the search engine; and

presenting the search results associated with the user-submitted query; and

in an event that the user-submitted query is a difficult query;

identifying, without further input from the user, a plurality of candidate queries;

for each candidate query, without further input from the user;

generating a feature vector that includes three or more features, each of the three or more features being individual components of the feature vector, each feature reflecting a measurement of effectiveness of the candidate query with respect to the user-submitted query, the measurement of effectiveness of the candidate query based, at least in part, on search results of the candidate query, the three or more features including an estimated normalized discounted cumulative gain; and

calculating a usefulness probability value based at least in part on the feature vector, wherein the usefulness probability value reflects an estimated quality of the search results for the candidate query with respect to the user-submitted query;

rank ordering the candidate queries based on the usefulness probability values; and

suggesting one or more of the top-ranked candidate queries as alternate queries for the user-submitted query;

in the event that the user-submitted query is a difficult query;

receiving a user selection of a particular query selected from the user-submitted query and the alternate queries that are suggested; and

presenting the search results associated with the particular query.

11. A method as recited in claim 9, wherein determining whether the user-submitted query is a difficult query comprises:

estimating a difficulty value associated with the user-submitted query, wherein the difficulty value represents a degree to which the user-submitted query is expected to return relevant or authoritative results; and

comparing the difficulty value to a threshold value.

12. One or more non-transitory computer storage media encoded with computer-executable instructions that, when executed, configure a computer system to perform a method as recited in claim 9.

13. A method comprising:

receiving a user-submitted query originating from a user;

determining whether the user-submitted query is a difficult query, wherein a difficult query is a query that will not return relevant or authoritative results;

in an event that the user-submitted query is not a difficult query;

determining search results associated with the user-submitted query; and

presenting the search results associated with the user-submitted query; and

in an event that the user-submitted query is a difficult query;

identifying a plurality of candidate queries based on the user-submitted query;

for each candidate query of the plurality of candidate queries, determining, without further input from the user, a feature vector having three or more features as individual components of the feature vector, each feature reflecting a measurement of effectiveness of the candidate query with respect to the user-submitted query, the measurement of effectiveness of the candidate query based, at least in part, on search results of the candidate query, the three or more features including an estimated normalized discounted cumulative gain, and any combination of;

one or more match features that reflect how well terms in the candidate query match search results of the candidate query, one or more cross-match features that reflect how well terms in the user-submitted query match search results of the candidate query, and one or more similarity features that reflect a degree of similarity between the search results of the user-submitted query and the search results of the candidate query;

rank ordering the plurality of candidate queries based at least in part on the feature vector of the respective candidate queries of the plurality of candidate queries; and

suggesting, based on the rank ordering, one or more of the candidate queries as alternate queries for the user-submitted query.

14. A method a recited in claim 13, wherein determining the three or more features includes determining in addition to a value of the estimated normalized discounted cumulative gain, a value for each of a title match, a snippet match, a uniform resource locator (URL) match, a title cross match, a snippet cross match, a URL cross match, a page similarity, an URL similarity, and a domain similarity.

15. A method as recited in claim 13, wherein rank ordering the candidate queries is further based, at least in part, on a value of the estimated normalized discounted cumulative gain calculated based, at least in part, on an estimated relevance rating.