Bot detection using profile-based filtration

• US 20130198203A1
• Filed: 12/22/2011
• Published: 08/01/2013
• Est. Priority Date: 12/22/2011
• Status: Abandoned Application

• Current Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Bates, John, Robison, Ben S.
• Application Number:
  US13/335,710
• Publication Number:
  US 20130198203A1
• Time in Patent Office: Days
• US Class Current: 707/748
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0241 Advertisements

Abstract

Methods and apparatus for bot detection using profile based filtration are disclosed. A statistical profile describing attributes of automated-origin content request activity for a network content provider is built. A plurality of content requests of unknown origin in terms of similarity to the attributes is scored. A likelihood of automated-origin content request activity based on the scoring is indicated.

Claims

1. A method, comprising:

building a statistical profile describing attributes of automated-origin content request activity for a network content provider;

scoring a plurality of content requests of unknown origin in terms of similarity to the attributes according to the statistical profile; and

indicating a likelihood of automated-origin content request activity based on the scoring.

2. The method of claim 1, further comprising,

prior to the scoring, filtering the plurality of content requests of unknown origin to eliminate from said scoring ones of the plurality of content requests of unknown origin arriving from a list of known automated sources.

3. The method of claim 1, wherein,

the building the statistical profile of attributes of automated content request activity further comprises characterizing a plurality of content requests arriving from a list of known automated sources.

4. The method of claim 3,

the building the statistical profile of attributes of automated content request activity further comprises characterizing a plurality of content requests arriving from a list of known automated sources for common attributes.

5. The method of claim 3,

the building the statistical profile of attributes of automated content request activity further comprises characterizing a plurality of content requests arriving from a list of known automated sources for attributes dissimilar from attributes of known non-automated content request activity.

6. The method of claim 1, further comprising:

generating analytics reports describing features of a set of content requests selected to exclude content requests having a high likelihood of automated-origin content request activity.

7. The method of claim 1, wherein

attributes of automated-origin content request activity comprise a number of page views in a particular period of time.

8. The method of claim 1, wherein

attributes of automated-origin content request activity comprise pre-fetching of content in advance of a user request.

9. The method of claim 1, wherein,

the building the statistical profile of attributes of automated content request activity further comprises characterizing a plurality of content requests arriving from a list of known non-automated sources.

10. The method of claim 1, wherein

the building the statistical profile describing attributes of automated-origin content request activity for a network content provider further comprises assigning weights to respective attributes based on relative correlation strength between a value of an attribute and a likelihood of automated activity;

and

the scoring a plurality of content requests of unknown origin in terms of similarity to the attributes further comprises applying the weights.

11. The method of claim 1, wherein

the method further comprises updating attributes of automated-origin content request activity by scoring a plurality of content requests arriving from automated sources identified after the building the statistical profile.

12. The method of claim 1, wherein

the method further comprises updating attributes of automated-origin content request activity by scoring using a logistical regression approach.

13. The method of claim 1, wherein

the method further comprises updating attributes of automated-origin content request activity by scoring using a neural networks approach.

14. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are computer-executable to implement:

building a statistical profile describing attributes of automated-origin content request activity for a network content provider;

scoring a plurality of content requests of unknown origin in terms of similarity to the attributes according to the statistical profile; and

indicating a likelihood of automated-origin content request activity based on the scoring.

15. The non-transitory computer-readable storage medium of claim 14, further comprising program instructions computer-executable to implement:

filtering the plurality of content requests of unknown origin to eliminate from the scoring ones of the plurality of content requests of unknown origin arriving from a list of known automated sources.

16. The non-transitory computer-readable storage medium of claim 14, wherein:

the program instructions computer-executable to implement;

updating attributes of automated-origin content request activity by characterizing using a neural networks approach a plurality of content requests arriving from automated sources identified after the building the statistical profile.

17. A system, comprising:

at least one processor; and

a memory comprising program instructions, wherein the program instructions are executable by the at least one processor to;

build a statistical profile describing attributes of automated-origin content request activity for a network content provider;

score a plurality of content requests of unknown origin in terms of similarity to the attributes according to the statistical profile; and

designate a sample of the content requests selected as having a low likelihood of automated-origin content request activity based on the scoring.

18. The system of claim 17, further comprising program instructions executable by the at least one processor to:

filter the plurality of content requests of unknown origin to eliminate from the scoring ones of the plurality of content requests of unknown origin arriving from a list of known automated sources.

19. The system of claim 17, further comprising program instructions executable by the at least one processor to generate analytics reports describing features of the sample of the content requests selected as having a low likelihood of automated-origin content request activity based on the scoring.

20. The system of claim 17, further comprising program instructions executable by the at least one processor to:

update attributes of automated-origin content request activity by characterizing using a neural networks approach a plurality of content requests arriving from automated sources identified after the building the statistical profile.