Extraction of rich search information from index servers via an alternative asynchronous data path

• US 8,306,964 B2
• Filed: 07/20/2010
• Issued: 11/06/2012
• Est. Priority Date: 07/20/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yu, Yinzhe, Zou, Qinghua, Pei, Pei, Sareen, Gaurav, Tan, Wee Wah Alfian
• Primary Examiner(s): DANG, THANH HA T
• Application Number:
  US12/840,281
• Publication Number:
  US 20120023093A1
• Time in Patent Office: 840 Days
• Field of Search: 707/706, 707/711, 707/723, 707/741, 707/753
• US Class Current: 707/706
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A search engine system is described herein that provides an alternative data path for collecting results provided by index servers. The alternative data path collects the results in a direct and asynchronous manner; this is in contrast to a synchronous path used to deliver search results to end users via one or more aggregator modules. An analysis system can use the alternative data path to collect a large amount of richly descriptive information regarding the performance of the search engine system, circumventing bottlenecks and other constraints that would otherwise be imposed by the synchronous data path. The analysis system can analyze the information collected from the index servers to improve the performance of the search engine system.

Claims

1. A method, using a search engine system, for providing results to an analysis system, comprising:

receiving at least one query that has been prepared to investigate at least one characteristic of the search engine system;

supplying said at least one query to plural index servers;

generating final results using the plural index servers based on said at least one query; and

sending said final results from the plural index servers to the analysis system using an alternative data path, the alternative data path being different from a synchronous data path used to send search results to end users,

wherein said generating of the final results comprises, with respect to each index server;

providing original intermediate results that include plural original result items, the plural original result items identifying respective documents using original document identifiers; and

transforming the original intermediate results into an instance of the final results by converting the original document identifiers into final document identifiers, by;

ordering the plural original result items based on the original document identifiers, to provide ordered intermediate results;

converting the original document identifiers in the ordered intermediate results into the final document identifiers, based on an order specified in the ordered intermediate results, to provide transformed results; and

changing the order of the transformed results back into an original order specified by the original intermediate results, to provide the instance of the final results.

2. The method of claim 1, wherein the synchronous data path involves use of at least one aggregator module that aggregates search results provided by the plural index servers, and wherein the alternative data path does not use any aggregator modules.

3. The method of claim 1, wherein the alternative data path is an asynchronous data path.

4. The method of claim 1, wherein said at least one query comprises a batch that includes plural queries.

5. The method of claim 4, wherein each query in the batch of queries includes a query identifier and a batch identifier.

6. The method of claim 1, wherein the final document identifiers comprise universal resource locator identifiers.

7. The method of claim 1, wherein at least one original result item in the original intermediate results is a multi-line result item that includes plural lines, and wherein said transforming comprises:

extracting an original document identifier from the multi-line result item;

transforming the original document identifier into a final document identifier;

extracting feature information from the multi-line result item; and

assembling the final document identifier and the feature information into a single-line result item of the final results.

8. The method of claim 1, wherein said sending comprises, for each index server:

waiting a specified time to send an instance of the final results to the analysis system; and

at the specified time, sending the instance of the final results to the analysis system via the alternative data path, wherein specified times for the plural index servers differ from each other.

9. The method of claim 1, further comprising, at a job completion determination module:

receiving the final results provided by plural index server modules; and

analyzing the final results to determine if all the final results have been received for an identified batch of queries.

10. An index server, comprising:

a central processor unit (CPU);

a result generator module configured to provide original intermediate results that include plural original result items, the plural original result items identifying respective documents using original document identifiers;

a post-processor module configured to transform the original intermediate results into final results by converting the original document identifiers into final document identifiers; and

an uploader module configured to send the final results to an analysis system using an alternative data path, the alternative data path being different from a synchronous data path used to send search results to end users,

wherein the post-processor module comprises;

logic configured to order the plural original result items based on the original document identifiers, to provide ordered intermediate results;

logic configured to convert the original document identifiers in the ordered intermediate results into the final document identifiers, based on an order specified in the ordered intermediate results, to provide transformed results; and

logic configured to change the order of the transformed results back into an original order specified by the original intermediate results, to provide the final results.

11. The index server of claim 10, wherein the alternative data path is an asynchronous data path.

12. The index server of claim 10, wherein the original document identifiers comprise internal document identifiers used by the index server to identified respective documents, and wherein the final document identifiers comprise uniform resource locator identifiers.

13. The index server of claim 10, wherein at least one original result item in the original intermediate results is a multi-line result item that includes plural lines, and wherein said post-processor comprises:

logic configured to extract an original document identifier from the multi-line result item;

logic configured to transform the original document identifier into a final document identifier;

logic configured to extract feature information from the multi-line result item; and

logic configured to assemble the final document identifier and the feature information into a single-line result item of the final results.

14. The index server of claim 10, wherein the uploader module comprises logic configured to send the final results to the analysis system via the alternative data path at a specified time, wherein specified times for plural index servers differ from each other.

15. A computer usable memory storing computer readable instructions, the computer readable instructions providing index server logic when executed by one or more processing devices, the computer readable instructions comprising:

logic configured to provide original intermediate results that include plural original result items, the plural original result items identifying respective documents using original document identifiers;

logic configured to order the plural original result items based on the original document identifiers, to provide ordered intermediate results;

logic configured to convert the original document identifiers in the ordered intermediate results into final document identifiers, based on an order specified in the ordered intermediate results, to provide transformed results;

logic configured to change the order of the transformed results back into an original order specified by the original intermediate results, to provide final results; and

logic configured to send the final results to an analysis system using an alternative data path, the alternative data path being different from a synchronous data path used to send search results to end users via at least one accumulator module.

16. The computer usable memory of claim 15, wherein at least one original result item in the original intermediate results is a multi-line result item that includes plural lines.

17. The computer usable memory of claim 15, wherein said logic configured to send is operative to send the final results in a staggered manner with respect to other index server logic employed by other respective index servers.