Detection of improper search queries in a wide area network search engine

• US 20040220915A1
• Filed: 05/02/2003
• Published: 11/04/2004
• Est. Priority Date: 05/02/2003
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Biggs, Jody D., Kline, Scott B.
• Granted Patent: US 8,166,014 B2
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9532 Query formulation

Abstract

Search queries which are used to assess performance of search listings are filtered such that the remaining search queries more accurately represent behavior of human users genuinely looking for information in a search engine. Search queries which come from unrecognized user agents or which come from the same network address too closely together or too regularly are removed. If a volume of search queries for a given search term in a period of time exceeds an expected volume, search queries for that term are discarded.

Claims

1. A method for detecting illegitimate search queries, the method comprising:

for each subject one of one or more received search queries;

identifying a user agent from which the subject search query was submitted;

comparing the user agent to one or more predetermined legitimate user agents;

upon a condition in which the user agent does not match any of the predetermined legitimate user agents, marking the subject search query as illegitimate.

2. The method of claim 1 wherein identifying comprises:

retrieving data representing an environmental variable associated with form data submitted by a user in conjunction with submitting the subject search query.

3. The method of claim 1 further comprising:

removing search queries marked as illegitimate from a database of search queries to thereby make the database more accurately representative of human user search behavior.

4. A method for detecting illegitimate search queries, each of which is received from a respective network address, the method comprising:

determining that one or more selected ones of the search queries were each submitted within less than a predetermined period of time of submission of a respective earlier-submitted one of the search queries, wherein the selected search queries and the respective earlier-submitted search queries are received from the same network address;

marking the selected search queries as illegitimate.

5. The method of claim 4 wherein determining comprises:

determining from which network address the search queries are received by retrieving data representing an environmental variable associated with form data submitted by a user in conjunction with submitting the subject search query.

6. The method of claim 4 further comprising:

removing the selected search queries from a database of search queries to thereby make the database more accurately representative of human user search behavior.

7. A method for detecting illegitimate search queries, each of which is received from a respective network address, the method comprising:

determining that one or more selected ones of the search queries were each submitted at time intervals which vary less than a predetermined amount, wherein the selected search queries and the respective earlier-submitted search queries are received from the same network address;

marking the selected search queries as illegitimate.

8. The method of claim 7 wherein determining comprises:

determining from which network address the search queries are received by retrieving data representing an environmental variable associated with form data submitted by a user in conjunction with submitting the subject search query.

9. The method of claim 7 further comprising:

removing the selected search queries from a database of search queries to thereby make the database more accurately representative of human user search behavior.

10. The method of claim 7 wherein determining comprises:

determining a standard deviation of the time intervals between receipt of the selected search queries.

11. A method for improving the accuracy by which a database of search queries represent human search behavior, the method comprising:

comparing a quantity of the search queries to an expected quantity of search queries;

determining that the quantity of search queries exceeds the expected quantity of search queries by at least a predetermined margin;

selecting a number of search queries of the quantity in accordance with an amount by which the quantity exceeds the expected quantity; and

removing the selected number of search queries from the quantity.

12. The method of claim 11 wherein the quantity and the expected quantity correspond to search queries which include a particular search term.

13. The method of claim 11 wherein the quantity and the expected quantity correspond to search queries specific to a particular geographic region.

14. The method of claim 11 wherein the quantity is sampled over a window of time.

15. The method of claim 14 wherein the window of time is fifteen minutes.

16. The method of claim 14 wherein the window of time is twenty-four hours.

17. The method of claim 11 further comprising:

determining the expected quantity of search queries.