Method for estimating coverage of Web search engines

• US 20050055342A1
• Filed: 01/21/2004
• Published: 03/10/2005
• Est. Priority Date: 11/08/2000
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Andrei Zary Broder, Krishna Asur Bharat
• Inventors: Broder, Andrei Zary, Bharat, Krishna Asur
• Granted Patent: US 7,308,445 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A computerized method is used to estimate the relative coverage of Web search engines. Each search engine maintains an index of words of pages located at specific URL addresses in a network. The method generates a random query. The random query is a logical combination of words found in a subset of the pages. The random query is submitted to a first search engine. In response a set of URLs of pages matching the query are received. Each URL identifies a page indexed by the first search engine that satisfies the random query. A particular URL identifying a sample page is randomly selected. A strong query corresponding to the sample page is generated, and the strong query is submitted to a second search engine. Result information received in response to the strong query is compared to determine if the second search engine has indexed the sample page, or a page substantially similar to the sample page. This procedure is repeated to gather statistical data which is used to estimate the relative sizes and amount of overlap of search engines.

Claims

1. A computerized method for estimating coverage of search engines, each search engine maintaining an index of words of pages located at specific address in a network, comprising the steps of:

generating a random query, the random query being a logical combination of words found in a training set of the pages;

submitting the random query to a first search engine;

receiving a set of URLs in response to the random query;

randomly selecting a particular URL identifying a sample page;

generating a strong query for the sample page;

submitting the strong query to a second search engine; and

comparing result information received in response to the strong query to determine if the second search engine has indexed the sample page.