Systems and Methods to Evaluate Search Qualities

• US 20100174736A1
• Filed: 01/06/2009
• Published: 07/08/2010
• Est. Priority Date: 01/06/2009
• Status: Active Grant

• Current Assignee: AT&T, Inc., AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T, Inc., AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: KUMAR, Ritu, JACOBSON, Guy Joseph, GOODALL, Colin R., HSU, Wen-Ling, MOHS, Bradley Keith, HALASZ, Sylvia, SKUDLARK, Ann E.
• Granted Patent: US 9,519,712 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Systems and methods are provided to evaluate a search engine using search results retrieved from a plurality of different search engines. For example, a controller can randomly distribute a set of pre-formulated queries to a plurality of computers to query the search engines at random times to obtain search results and combine the search results to evaluate the completeness of a query result provided by a particular search engine. The controller may simulate the random queries from users using a set of computers and may reduce the burden on the search engines by spreading out the query activities over a period of time, such as a month.

Claims

1. A method, comprising:

a controller dividing a set of predetermined queries into a plurality of batches;

the controller communicating with a plurality of computers to cause the plurality of computers to submit queries in the batches to a plurality of separate search engines at random times during a predetermined period of time; and

determining a performance indicator of at least one of the search engines based on results received in response to the queries submitted to the plurality of search engines.

2. The method of claim 1, wherein the determining of the performance indicator comprises:

combining the results received in response to the queries submitted to the plurality of search engines to generate a combined set of results; and

evaluating a completeness indicator of a query submitted to one of the search engines using the combined set of results.

3. The method of claim 2, wherein the combining of the results comprises:

normalizing results received from the plurality of search engines to generate normalized results; and

identifying unique results from the normalized results to form the combined set of results.

4. The method of claim 3, wherein the normalized results include at least telephone number;

and the unique results are identified based at least in part on telephone number.

5. The method of claim 4, wherein the normalized results further include address and name.

6. The method of claim 5, wherein the unique results are further identified based at least in part on name and address.

7. The method of claim 3, wherein the combining of the results further comprises:

eliminating invalid results based on telephone number.

8. The method of claim 7, wherein the eliminating of invalid results further comprises:

extracting data from a telephone number field of a query result;

determining whether the data is a valid telephone number; and

eliminating the query result if the data is not a valid telephone number.

9. The method of claim 8, wherein the determining of whether the data is a valid telephone number comprises determining whether the telephone number has a proper exchange number.

10. The method of claim 9, wherein the determining of whether the data is a valid telephone number further comprises determining a most recent date of the telephone number being used for a call.

11. The method of claim 10, wherein the determining of whether the data is a valid telephone number further comprises initiating a call routing signal to call the telephone number to determine whether the telephone number is in service.

12. The method of claim 8, further comprising:

determining a rate of invalid results for at least one search engine based on identifying invalid telephone numbers in query results.

13. The method of claim 1, wherein the controller communicating with a plurality of computers comprises:

the controller randomly selecting a computer from the plurality of computers to execute one batch of the plurality of batches.

14. The method of claim 13, wherein the batch includes a plurality of queries, and the method further includes:

specifying random time intervals between the plurality of queries in the batch.

15. The method of claim 13, wherein the batch is randomly selected from the plurality of batches for the computer.

16. The method of claim 1, further comprising:

receiving queries from end users at one of the search engines; and

determining the set of predetermined queries based on the queries received from the end users.

17. The method of claim 1, wherein the determining of the performance indicator comprises:

combining the results received in response to the queries submitted to the plurality of search engines to generate a combined set of results; and

evaluating a completeness indicator of a group of queries submitted to one of the search engines using the combined set of results.

18. The method of claim 17, wherein the group of queries is grouped based on a predetermined geographic search criterion.

19. A machine readable media storing instructions, the instructions causing a controller to perform a method, the method comprising:

the controller dividing a set of predetermined queries into a plurality of batches;

the controller communicating with a plurality of computers to cause the plurality of computers to submit queries in the batches to a plurality of separate search engines at random times during a predetermined period of time; and

determining a performance indicator of at least one of the search engines based on results received in response to the queries submitted to the plurality of search engines.

20. A system, comprising:

means for dividing a set of predetermined queries into a plurality of batches via a controller;

means for causing a plurality of computers to submit queries in the batches to a plurality of separate search engines at random times during a predetermined period of time; and

means for determining a performance indicator of at least one of the search engines based on results received in response to the queries submitted to the plurality of search engines.