Detecting spam related and biased contexts for programmable search engines

• US 7,743,045 B2
• Filed: 08/10/2005
• Issued: 06/22/2010
• Est. Priority Date: 08/10/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Guha, Ramanathan V.
• Primary Examiner(s): Wong; Don
• Assistant Examiner(s): Brown; Sheree N
• Application Number:
  US11/202,382
• Publication Number:
  US 20070038600A1
• Time in Patent Office: 1,777 Days
• Field of Search: 707 3- 5, 707/1
• US Class Current: 707/706
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/0256 User search

Abstract

A programmable search engine system is programmable by a variety of different entities, such as client devices and vertical content sites to customize search results for users. Context files store instructions for controlling the operations of the programmable search engine. The context files are processed by various context processors, which use the instructions therein to provide various pre-processing, post-processing, and search engine control operations. Spam related and biased contexts and search results are identified using offline and query time processing stages, and the context files from vertical content providers associated with such spam and biased context and results are excluded from processing on direct user queries.

Claims

1. A computer-implemented method, comprising:

maintaining in a data processing system a collection of context files for use by a search engine system, each context file provided by a third-party content provider to the search engine system, each context file containing commands to control operation of the search engine system in processing search query inputs;

receiving from a plurality of third-party content providers a plurality of new context files for addition to the collection, each of the providers providing at least one of the new context files, each new context file provided by the respective third-party content provider to the search engine system each new context file containing commands to control operation of the search engine system in processing search query inputs;

for a first new context file, determining that a number of spam web pages listed in the first new context file exceeds a threshold number and consequently not adding the first new context file to the collection;

receiving in a search engine a search query input from a user, the search query input having been received through an interface provided to the user by the third-party content provider;

identifying a first context file from the collection for processing the search query input;

using commands in the first context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the first context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query; and

providing the context processed search results in accordance with the commands in the first context file, wherein the identifying, processing, generating and providing are performed by one or more processors.

2. The method of claim 1, wherein at least one context file in the collection includes links to related context files.

3. The method of claim 1, further comprising:

executing at least one command from the first context file, the command selected from a group of commands comprising;

a query evaluation command for evaluating a property value of a query with respect to a predefined property;

a query modification command for modifying the search query input;

a related context command for identifying a related context file in the context processed search results; and

a redirection command for identifying a related context file for processing.

4. The method of claim 1, further comprising:

executing at least one command from the first context file, the command selected from a group of commands comprising;

a corpus selection command for selecting a document collection for searching with the search query input; and

a search engine parameter command for establishing a parameter of a search engine for processing the search query input.

5. The method of claim 1, further comprising:

executing at least one command from the first context file, the command selected from a group of commands comprising;

a restriction command for filtering the search results to include only search results of a designated type in the context processed search results; and

an annotation command for annotating the context processed search results.

6. The method of claim 1, wherein the context processed search results are presented with annotations specified in the first context file.

7. The method of claim 6, wherein the annotations comprise links to web pages.

8. The method of claim 6, wherein the annotations comprise labels, further comprising:

receiving a selection of one of the labels; and

generating new search results, wherein the new search results are associated with metadata that include the label.

9. A computer-implemented method, comprising:

receiving in a search engine a search query input from a user, the search query input having been input received through an interface provided to the user by a third-party content provider;

identifying a context file provided by the third-party content provider, the context file containing commands for processing the search query input;

using the commands in the context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query;

determining that a spam measure of the context processed search results exceeds a spam threshold; and

excluding spam search results, links, annotations, or other context related content included in the context file from the context processed search results and providing the resulting context processed search results in accordance with the commands in the context file, wherein the identifying, processing, generating and excluding are performed by one or more processors.

10. The method of claim 9, wherein the context file includes links to related context files.

11. The method of claim 9, further comprising:

executing at least one command from the context file, the command selected from a group of commands comprising;

a query evaluation command for evaluating a property value of a query with respect to a predefined property;

a query modification command for modifying the search query input;

a related context command for identifying a related context file in the context processed search results; and

a redirection command for identifying a related context file for processing.

12. A computer-implemented method comprising:

receiving in a search engine a search query input from a user through an interface provided to the user by a third-party content provider;

identifying a context file provided by the third-party content provider, the context file containing commands for processing the search query input;

using the commands in the context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query;

generating native processed search results responsive to the search query input;

calculating a distance measure between the context processed search results and the native processed search results;

determining that the distance measure exceeds a distance threshold and consequently removing, from the context processed search results, context processed search results that are not found in the native search results to produce filtered context processed search results; and

providing the filtered context processed search results in response to the search query input, wherein the identifying, processing, and providing are performed by one or more processors.

13. The method of claim 12, wherein the distance measure is a percentage of the context processed search results that are included in the native search results.

14. A system comprising:

one or more computers;

a computer-readable medium coupled to the one or more computers having instructions stored thereon executable by the one or more computers to cause the one or more computers to perform operations comprising;

maintaining in a data processing system a collection of context files for use by a search engine system, each context file provided by a third-party content provider to the search engine system, each context file containing commands to control operation of the search engine system in processing search query inputs;

receiving from a plurality of third-party content providers a plurality of new context files for addition to the collection, each of the providers providing at least one of the new context files, each new context file provided by the respective third-party content provider to the search engine system, each new context file containing commands to control operation of the search engine system in processing search query inputs;

for a first new context file, determining that the number of spam web pages listed in the first new context file exceeds a threshold number and consequently not adding the first new context file to the collection;

receiving in a search engine a search query input from a user, the search query input having been received through an interface provided to the user by the third-party content provider;

identifying a first context file from the collection for processing the search query input;

using commands in the first context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the first context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query; and

providing the context processed search results in accordance with the commands in the first context file.

15. A system comprising:

one or more computers;

a computer-readable medium coupled to the one or more computers having instructions stored thereon executable by the one or more computers to cause the one or more computers to perform operations comprising;

receiving in a search engine a search query input from a user, the search query input having been input received through an interface provided to the user by a third-party content provider;

identifying a context file provided by the third-party content provider, the context file containing commands for processing the search query input;

using the commands in the context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query;

determining that a spam measure of the context processed search results exceeds a spam threshold;

including context content in the context processed search results provided to the user if the spam measure does not exceed the spam threshold; and

excluding spam search results, links, annotations, or other context related content included in the context file from the context processed search results and providing the resulting context processed search results in accordance with the commands in the context file.

16. The system of claim 15, wherein the operations further comprise:

transmitting the search results to a client device.

17. A system comprising:

one or more computers;

a computer-readable medium coupled to the one or more computers having instructions stored thereon executable by the one or more computers to cause the one or more computers to perform operations comprising;

receiving in a search engine a search query input from a user through an interface provided to the user by a third-party content provider;

identifying a context file provided by the third-party content provider, the context file containing commands for processing the search query input;

using the commands in the context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query;

generating native processed search results responsive to the search query input;

calculating a distance measure between the context processed search results and the native processed search results;

determining that the distance measure exceeds a distance threshold and consequently removing, from the context processed search results, context processed search results that are not found in the native search results to produce filtered context processed search results; and

providing the filtered context processed search results in response to the search query input.

18. The system of claim 17, wherein the operations further comprise:

transmitting the filtered context processed search results to a client device.

19. A computer-readable medium encoded with a computer program, the computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

maintaining in a data processing system a collection of context files for use by a search engine system, each context file provided by a third-party content provider to the search engine system, each context file containing commands to control operation of the search engine system in processing search query inputs;

receiving from a plurality of third-party content providers a plurality of new context files for addition to the collection, each of the providers providing at least one of the new context files, each new context file provided by the respective third-party content provider to the search engine system each new context file containing commands to control operation of the search engine system in processing search query inputs;

for a first new context file, determining that a number of spam web pages listed in the first new context file exceeds a threshold number and consequently not adding the first new context file to the collection;

receiving in a search engine a search query input from a user, the search query input having been received through an interface provided to the user by the third-party content provider;

identifying a first context file from the collection for processing the search query input;

using commands in the first context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the first context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query; and

providing the context processed search results in accordance with the commands in the first context file, wherein the identifying, processing, generating and providing are performed by one or more processors.

20. A computer-readable medium encoded with a computer program, the computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

receiving in a search engine a search query input from a user, the search query input having been input received through an interface provided to the user by a third-party content provider;

identifying a context file provided by the third-party content provider, the context file containing commands for processing the search query input;

using the commands in the context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query;

determining that a spam measure of the context processed search results exceeds a spam threshold; and

excluding spam search results, links, annotations, or other context related content included in the context file from the context processed search results and providing the resulting context processed search results in accordance with the commands in the context file, wherein the identifying, processing, generating and excluding are performed by one or more processors.

21. A computer-readable medium encoded with a computer program, the computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

receiving in a search engine a search query input from a user through an interface provided to the user by a third-party content provider;

identifying a context file provided by the third-party content provider, the context file containing commands for processing the search query input;

using the commands in the context file to control an organization and a presentation of search results resulting from the processing of the search query input, including;

processing the search query input using the commands in the context file to produce a context processed search query;

generating context processed search results responsive to the context processed search query;

generating native processed search results responsive to the search query input;

calculating a distance measure between the context processed search results and the native processed search results;

determining that the distance measure exceeds a distance threshold and consequently removing, from the context processed search results, context processed search results that are not found in the native search results to produce filtered context processed search results; and

providing the filtered context processed search results in response to the search query input, wherein the identifying, processing, and providing are performed by one or more processors.