×

Systems and methods of handling internet spiders

  • US 7,987,173 B2
  • Filed: 07/30/2010
  • Issued: 07/26/2011
  • Est. Priority Date: 08/25/2009
  • Status: Expired due to Fees
First Claim
Patent Images

1. A method for implementation on one or more computers, comprising:

  • accepting, at one or more of the one or more computers, requests for resources, each resource identified by a respective URL resolvable to a common domain;

    collecting, at one or more of the one or more computers, identifying information for entities making the requests;

    comparing, at one or more of the one or more computers, the identifying information for the entities making the requests; and

    categorizing, at one or more of the one or more computers, one or more of the entities as a web crawler, based on determining whether at least a threshold number of requests for the identical URL were made from a single entity within a time limit, and responsive to the determining returning an affirmative result, categorizing the entity that made those requests as a web crawler.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×