Detecting spam related and biased contexts for programmable search engines

• US 20070038600A1
• Filed: 08/10/2005
• Published: 02/15/2007
• Est. Priority Date: 08/10/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Guha, Ramanathan
• Granted Patent: US 7,743,045 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/0256 User search

Abstract

A programmable search engine system is programmable by a variety of different entities, such as client devices and vertical content sites to customize search results for users. Context files store instructions for controlling the operations of the programmable search engine. The context files are processed by various context processors, which use the instructions therein to provide various pre-processing, post-processing, and search engine control operations. Spam related and biased contexts and search results are identified using offline and query time processing stages, and the context files from vertical content providers associated with such spam and biased context and results are excluded from processing on direct user queries.

Claims

1. A method of processing a search query for a search engine of a search engine system to provide a set of search results to the search query, the method comprising:

receiving at the search engine system a search query from a client device of a user;

identifying at least one context file for processing the search query, the context file including commands, each context file associated with a vertical content provider;

processing the search query with the context file to produce a context processed search query;

executing the context processed search query on a search engine obtain a set of search results;

processing the search results at the search engine system using the commands in the identified context file to produce a set of context processed search results, the context processed search results including at least one annotation of a search result the annotation provided by the context file, and at least one hyperlink to a second set of search results, the hyperlink provided by the context file;

filtering the context processed search results to remove spam search results; and

providing the filtered context processed search results to the client device of the user.

2. The method of claim 1, wherein filtering the context processed search results to remove spam search results further comprises:

determining whether the context processed search results include spam search results;

determining a vertical content provided associated with the spam search results;

removing the spam search results from the context processed search results; and

removing at least one annotation or at least one hyperlink contained in the context file of the determined vertical content provider from the context processed search results.

3. The method of claim 1, further comprising:

filtering the context processed search results to remove biased search results.

4. The method of claim 3, wherein filtering the context processed search results to remove biased search results comprises:

determining a distance measure between the context processed search results and search results of the received search query;

responsive to the distance measure exceeding a threshold;

determining the context processed search results to be biased, determining a vertical content provided associated with the context processed search results;

removing the biased search results from the context processed search results; and

removing at least one annotation or at least one hyperlink contained in the context file of the determined vertical content provider from the context processed search results.

5. A method of processing a search query for a search engine of a search engine system to provide a set of search results to the search query, the method comprising:

receiving at the search engine system a plurality of context files, each context file associated with a vertical content provider, and including commands;

processing each received context file with a spam filter to detect context file including spam related content;

identifying as spam related the vertical content provider associated with the context file including spam related content;

receiving at the search engine system a search query from a client device of a user;

identifying at least one cached context file for processing the search query, selectively excluding the context files associated with spam related vertical content providers;

processing the search query with the context file to produce a context processed search query;

executing the context processed search query on a search engine obtain a set of search results;

processing the search results at the search engine system using the commands in the identified context file to produce a set of context processed search results; and

providing the filtered context processed search results to the client device of the user.

6. The method of claim 5, wherein receiving at the search engine system a plurality of context files comprises:

crawling a plurality of vertical content provider sites and extracting context files therefrom.

7. The method of claim 5, wherein receiving at the search engine system a plurality of context files comprises:

receiving the context files via a registration interface.

9. The programmable search engine system of claim 6, further comprising:

a bias detection filter that filters the context processed search results to remove biased search results.

10. The programmable search engine system of claim 9, wherein the bias detection filter determines a distance measure between the context processed search results and search results of the received search query.

11. The programmable search engine system of claim 10, wherein the bias detection filter responsive to the distance measure exceeding a threshold, determines the context processed search results to be biased, determines a vertical content provided associated with the context processed search results, removes the biased search results from the context processed search results;

and removes at least one annotation or at least one hyperlink contained in the context file of the determined vertical content provider from the context processed search results.

8. A programmable search engine system, comprising:

a repository of cached context files, the context files including commands for pre-processing a search query, and post-processing search results from the search query, selected ones of the context files associated with vertical content provider sites;

a context server that receives an identifier of a vertical content provider site from which a search query is received and retrieves from repository at least one context file associated with the vertical content provider site;

a context processor that modifies the search query according to a pre-processing command in the retrieved context file;

a search engine the searches a document collection using the modified search query to produce context processed search results; and

a spam filter that filters the context processed search results to remove spam related results.