Prediction and information retrieval for intrinsically diverse sessions

• US 9,594,837 B2
• Filed: 02/26/2013
• Issued: 03/14/2017
• Est. Priority Date: 02/26/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Raman, Karthik, Bennett, Paul Nathan, Collins-Thompson, Kevyn Breca
• Primary Examiner(s): Leroux, Etienne
• Assistant Examiner(s): BIBBEE, JARED M
• Application Number:
  US13/776,717
• Publication Number:
  US 20140244610A1
• Time in Patent Office: 1,477 Days
• Field of Search: 707/706
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

Various technologies described herein pertain to predicting intrinsically diverse sessions and retrieving information for such intrinsically diverse sessions. Search results retrieved by a search engine responsive to executing a query are received. A query classifier can be employed to determine whether the query is intrinsically diverse or not intrinsically diverse based on one or more features of the query and session interaction properties. The query is intrinsically diverse when included in an intrinsically diverse session directed towards a task, where the query and disparate queries included in the intrinsically diverse session are directed towards respective subtasks of the task. An objective function can be evaluated based at least upon the query to compute an optimized value when the query is determined to be intrinsically diverse. The search results can be presented on a display screen according to the optimized value when the query is determined to be intrinsically diverse.

Claims

1. A method of presenting search results returned responsive to executing a query, comprising:

determining, by employing a query classifier, whether the query is intrinsically diverse or not intrinsically diverse, wherein the query is intrinsically diverse when included in an intrinsically diverse session, wherein the intrinsically diverse session is directed towards a task, and wherein queries included in the intrinsically diverse session are directed towards respective subtasks of the task;

receiving the search results retrieved by a search engine responsive to executing the query, wherein related queries correspond to the search results;

evaluating an objective function to compute an optimized value in response to the query being determined to be intrinsically diverse, wherein;

the objective function is based on relevance of differing search results to the query, the differing search results returned responsive to the related queries; and

the objective function is based on redundancy between the related queries; and

controlling presentation of the search results on a display screen according to the optimized value when the query is determined to be intrinsically diverse.

2. The method of claim 1, wherein the objective function is further based on relevance of the search results to one or more of the related queries.

3. The method of claim 2, wherein the objective function is further based on relevance of the search results to the query.

4. The method of claim 1, wherein controlling the presentation of the search results on the display screen according to the optimized value when the query is determined to be intrinsically diverse further comprises:

selecting one or more of the related queries to be selected related queries, the selected related queries being selected based on the optimized value; and

causing the search results and the selected related queries to be displayed on the display screen.

5. The method of claim 1, wherein controlling the presentation of the search results on the display screen according to the optimized value when the query is determined to be intrinsically diverse further comprises:

selecting one or more of the related queries to be selected related queries, the selected related queries being selected based on the optimized value;

determining relevance measures of the search results to the selected related queries based on the optimized value; and

causing the search results and the selected related queries to be displayed on the display screen, wherein the selected related queries are positioned relative to the search results on the display screen based upon the relevance measures of the search results to the selected related queries.

6. The method of claim 1, wherein controlling the presentation of the search results on the display screen according to the optimized value when the query is determined to be intrinsically diverse further comprises:

re-ranking the search results retrieved by the search engine responsive to executing the query, the search results being re-ranked based on the optimized value; and

causing the search results to be displayed on the display screen as re-ranked.

7. The method of claim 1, wherein controlling the presentation of the search results on the display screen according to the optimized value when the query is determined to be intrinsically diverse further comprises:

selecting one or more of the related queries to be selected related queries, the selected related queries being selected based on the optimized value;

determining relevance measures of the search results to the selected related queries based on the optimized value;

re-ranking the search results retrieved by the search engine responsive to executing the query, the search results being re-ranked based on the optimized value; and

causing the search results and the selected related queries to be displayed on the display screen, wherein the search results are displayed on the display screen as re-ranked, and wherein the selected related queries are positioned relative to the search results on the display screen based upon the relevance measures of the search results to the selected related queries.

8. The method of claim 1, further comprising:

identifying one or more features of the query; and

determining, by employing the query classifier, whether the query is intrinsically diverse or not intrinsically diverse based at least on the one or more features of the query.

9. The method of claim 1, further comprising:

identifying one or more features of the query; and

determining, by employing the query classifier, whether the query is one of an initiator query, a successor query, or a non-intrinsically diverse query based on the one or more features of the query, wherein a first query in the intrinsically diverse session is the initiator query, and wherein remaining queries in the intrinsically diverse session other than the initiator query are successor queries.

16. The method of claim 1, the redundancy between the related queries being based on similarities between snippets of the differing search results returned responsive to the related queries.

19. The method of claim 1, the redundancy between the related queries being based on similarities between query text of the related queries.

10. A system that populates an intrinsically diverse session database for training a query classifier, comprising:

a data repository that retains a search log that comprises queries inputted to a search engine and the intrinsically diverse session database;

one or more processors; and

memory that comprises computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

accessing the search log;

identifying intrinsically diverse sessions from the search log, the intrinsically diverse sessions being identified by filtering the queries in the search log to extract the intrinsically diverse sessions from the search log, filtering the queries to extract the intrinsically diverse sessions from the search log comprising;

removing sessions from the search log that comprise an initiator query and less than a minimum number of successor queries, wherein an initiator query is a first query of a session and remaining queries in the session other than the initiator query are successor queries;

removing sessions from the search log that comprise successor queries that return search results having less than a minimum overlap with search results returned by respective initiator queries; and

removing sessions from the search log that comprise successor queries having greater than a maximum syntactic similarity with respective initiator queries;

retaining the intrinsically diverse sessions in the intrinsically diverse session database; and

training the query classifier based upon features of the intrinsically diverse sessions from the intrinsically diverse session database, wherein the query classifier is trained to identify whether a query inputted to the search engine is an intrinsically diverse query.

11. The system of claim 10, filtering the queries to extract the intrinsically diverse sessions from the search log further comprising at least one of:

removing queries from the search log having a frequency greater than a threshold frequency;

merging satisfied click results across duplicated queries in the search log;

removing queries from the search log that are not manually entered;

removing sessions from the search log that have less than a minimum number of satisfied clicks;

removing sessions from the search log that include queries having lengths greater than a threshold length;

or

sorting sessions based on number of distinct successor queries.

12. The system of claim 10, wherein the features of the intrinsically diverse sessions comprise at least one of query text, a query statistic, a query natural language processing (NLP) feature, a query topic, a property of queries appearing in a common session, or a property of sessions in which a given query appears.

13. A computer-readable storage device including computer-executable instructions that, when executed by a processor, cause the processor to perform acts including:

identifying one or more features of a query;

determining, by employing a query classifier, whether the query is intrinsically diverse or not intrinsically diverse based on the one or more features of the query, wherein the query is intrinsically diverse when included in an intrinsically diverse session, wherein the intrinsically diverse session is directed towards a task, and wherein queries included in the intrinsically diverse session are directed towards respective subtasks of the task;

receiving search results retrieved by a search engine responsive to executing the query;

receiving related queries that correspond to the search results;

evaluating an objective function to compute an optimized value when the query is determined to be intrinsically diverse, wherein;

the objective function is based on relevance of differing search results to the query, the differing search results returned responsive to the related queries; and

the objective function is based on redundancy between the related queries; and

controlling presentation of the search results on a display screen according to the optimized value when the query is determined to be intrinsically diverse.

14. The computer-readable storage device of claim 13, wherein the query classifier is trained based upon features of intrinsically diverse sessions from an intrinsically diverse session database.

15. The computer-readable storage device of claim 13, wherein the one or more features of the query comprise at least one of text of the query, a statistic of the query, a natural language processing (NLP) feature of the query, a topic of the query, a property of queries appearing in a common session with the query, or a property of sessions in which the query appears.

17. The computer-readable storage device of claim 13, wherein:

the objective function is further based on relevance of the search results to one or more of the related queries; and

the objective function is further based on relevance of the search results to the query.

18. The computer-readable storage device of claim 13, wherein controlling the presentation of the search results on the display screen according to the optimized value when the query is determined to be intrinsically diverse further comprises:

selecting one or more of the related queries to be selected related queries, the selected related queries being selected based on the optimized value; and

causing the search results and the selected related queries to be displayed on the display screen.

20. The computer-readable storage device of claim 13, the redundancy between the related queries being based on similarities between snippets of the differing search results returned responsive to the related queries.