Client side human user indicator

• US 9,420,049 B1
• Filed: 06/30/2010
• Issued: 08/16/2016
• Est. Priority Date: 06/30/2010
• Status: Active Grant

• Current Assignee: F5 Networks, Inc. (F5 Incorporated)
• Original Assignee: F5 Networks, Inc. (F5 Incorporated)
• Inventors: Yna, Shlomo, Margalit, Orit, Serfaty, Beni, Talmor, Ron
• Primary Examiner(s): Kim, Hee Soo
• Application Number:
  US12/828,237
• Time in Patent Office: 2,239 Days
• Field of Search: 709/203, 709/229, 709/224
• US Class Current: 1/1
• CPC Class Codes: H04L 2463/144 Detection or countermeasures against botnets
• CPC Class Codes: H04L 63/0209 Architectural arrangements, e.g. perimeter networks or demilitarized zones
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/1483 service impersonation, e.g. phishing, pharming or web spoofing detection of rogue wireless access points H04W12/12
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A system and method for preventing web scraping which includes receiving a request between a web client and a web server for the web client to receive web content. A client side language script is injected into a response to be sent to the requesting web client, wherein the client side language script contains an event listener to detect a keystroke and/or a mouse movement at the web client. Information is collected from the client side language script relating to whether the keystroke and/or the mouse movement were detected. The web client is selectively allowed to access the web server to receive the web content based on the collected information.

Claims

1. A method comprising:

receiving, by a network traffic manager device, a request from a web client device to receive web content from a web server device;

placing, by the network traffic manager device, a client side language script into a response to the request received from the web server device and to be sent to the web client device, wherein the client side language script comprises an event listener to detect human activity at the web client device and an access flag to indicate when the web client device is granted access to the web server device;

receiving, by the network traffic manager device, collected information from the client side language script;

analyzing, by the network traffic management computing device, the collected information to determine when an indicator of human activity associated with the web client device are detected; and

sending, by the network traffic manager device, a subsequent received request to the web server device and setting the access flag to indicate that the web client device is granted access to the web server device for a predetermined number of subsequent received requests from the web client device upon determining from the analyzed collected information that the indicator of human activity associated with the web client device was detected.

2. The method of claim 1, further comprising:

applying, by the network traffic manager device, a computational challenge in the request and sending the request back to the web client device prior to the placement of the client side language script; and

receiving, by the network traffic manager device, a reply to the computational challenge from the web client device, wherein the placement of the client side language script occurs when the reply to the computational challenge indicates that the web client device supports JavaScript.

3. The method of claim 1, further comprising:

granting, by the network traffic manager device, access to the web server device and receives web content when the indicator of human activity at the web client device is detected in a session or when the web client device is on an authorized list.

4. The method of claim 1, further comprising:

assigning, by the network traffic manager device, a session identification for the web client device when placing the client language script into the response sent to the web client device, wherein the web client device is granted access to the web server device when the collected information matches the session identification.

5. The method of claim 1, further comprising:

preventing, by the network traffic manager device, the web client device from accessing the web server device for the predetermined number of transactions upon determining from the analyzed collected information that the indicator of human activity associated with the web client device was not detected.

6. The method of claim 1, further comprising:

monitoring, by the network traffic manager device, a number of page traversals received from the web client device over a predetermined period of time, to determine when the web client device is operated by a human.

7. The method of claim 1, further comprising:

inserting, by the network traffic manager device, a web page link not viewable by a human to determine when the web client device is not operated by a human.

8. The method of claim 1, wherein a subsequent request does not include collected information from the client side language script, and further comprising:

sending, by the network traffic manager device, the client side language script in individual responses to the web client device for a predetermined number of grace intervals.

9. A non-transitory computer readable medium having stored thereon instructions for preventing web scraping, comprising machine executable code which when executed by at least one processor, causes the processor to perform steps rising:

receiving a request from a web client device to receive web content from a web server device;

placing a client side language script into a response to the request received from the web server device and to be sent to the web client device, wherein the client side language script comprises an event listener to detect human activity at the web client device and an access flag to indicate when the web client device is granted access to the web server device;

receiving collected information from the client side language script;

analyzing the collected information to determine when an indicator of human activity associated with the web client device are detected; and

sending a subsequent received request to the web server device and setting the access flag to indicate that the web client device is granted access to the web server device for a predetermined number of subsequent received requests from the web client device upon determining from the analyzed collected information that the indicator of human activity associated with the web client device was detected.

10. The medium of claim 9, further having stored thereon instructions that when executed by the processor cause the processor to perform steps further comprising:

applying a computational challenge in the request and sending the request back to the web client device prior to the placement of the client side language script; and

receiving a reply to the computational challenge from the web client device, wherein the placement of the client side language script occurs when the reply to the computational challenge indicates that the web client device supports JavaScript.

11. The medium of claim 9, further having stored thereon instructions that when executed by the processor cause the processor to perform steps further comprising:

granting access to the web server device and receives web content when human activity at the web client device is detected in a session or when the web client device is on an authorized list.

12. The medium of claim 9, further having stored thereon instructions that when executed by the processor cause the processor to perform steps further comprising:

assigning a session identification for the web client device when placing the client language script into the response sent to the web client device, wherein the web client device is granted access to the web server device when the collected information matches the session identification.

13. The medium of claim 9, further having stored thereon instructions that when executed by the processor cause the processor to perform steps further comprising:

preventing the web client device from accessing the web server device for the predetermined number of transactions upon determining from the analyzed collected information that the indicator of human activity associated with the web client device was not detected.

22. The medium of claim 9, further having stored thereon instructions that when executed by the processor cause the processor to perform steps further comprising:

monitoring a number of page traversals received from the web client device over a predetermined period of time, to determine when the web client device is operated by a human.

23. The medium of claim 9, further having stored thereon instructions that when executed by the processor cause the processor to perform steps further comprising:

inserting a web page link not viewable by a human to determine when the web client device is not operated by a human.

24. The medium of claim 9, wherein the subsequent request does not include collected information from the client side language script, and further having stored thereon instructions that when executed by the processor cause the processor to perform steps further comprising:

sending client side language script in individual responses to the web client device for a predetermined number of grace intervals.

14. A network traffic manager device comprising:

one or more processors;

a memory coupled to the one or more processors which are configured to be capable of executing programmed instructions comprising and stored in the memory to;

receive a request from a web client device to receive web content from a web server device;

place a client side language script into a response to the request received from the web server device and to be sent to the web client device, wherein the client side language script comprises an event listener to detect human activity at the web client device and an access flag to indicate when the web client device is granted access to the web server device;

receive collected information from the client side language script;

analyzing the collected information to determine when an indicator of human activity associated with the web client device are detected; and

send a subsequent received request to the web server device and set the access flag to indicate that the web client device is granted access to the web server device for a predetermined number of subsequent received requests from the web client device upon determining from the analyzed collected information that the indicator of human activity associated with the web client device was detected.

15. The network traffic manager device of claim 14, wherein the processor is further configured to be capable of executing programmed instructions comprising and stored in the memory to:

grant access to the web server device and receives web content when a keystroke or mouse movement is detected or when the web client device is on an authorized list.

16. The network traffic manager device of claim 14, wherein the processor is further configured to be capable of executing programmed instructions comprising and stored in the memory to:

place a computational challenge in the request and send the request back to the web client device prior to the placement of the client side language script; and

receive a reply to the computational challenge from the web client device, wherein the placement of the client side language script occurs when the reply to the computational challenge indicates that the web client device supports JavaScript.

17. The network traffic manager device of claim 14, wherein the processor is further configured to be capable of executing programmed instructions comprising and stored in the memory to:

assign a session identification for the web client device when placing the client language script into the response sent to the web client device, wherein the web client device is granted access to the web server device when the collected information matches the session identification.

18. The network traffic manager device of claim 14, wherein the processor is further configured to be capable of executing programmed instructions comprising and stored in the memory to:

prevent the web client device from accessing the web server device for a predetermined number of unsafe intervals upon determining from the analyzed collected information that the indicator of human activity associated with the web client device was not detected.

19. The network traffic manager device of claim 14, wherein the processor is further configured to be capable of executing programmed instructions comprising and stored in the memory to:

monitor a number of page traversals received from the web client device over a predetermined period of time, to determine when the web client device is operated by a human.

20. The network traffic manager device of claim 14, wherein the processor is further configured to be capable of executing programmed instructions comprising and stored in the memory to:

insert a web page link not viewable by a human to determine when the web client device is not operated by a human.

21. The network traffic manager device of claim 14, wherein the subsequent request does not include collected information from the client side language script, and wherein the processor is further configured to be capable of executing programmed instructions comprising and stored in the memory to:

send client side language script in individual responses to the web client device for a predetermined number of grace intervals.