Internet-based proxy service to modify internet responses

• US 9,369,437 B2
• Filed: 11/04/2010
• Issued: 06/14/2016
• Est. Priority Date: 04/01/2010
• Status: Active Grant

• Current Assignee: CloudFlare Incorporated
• Original Assignee: CloudFlare Incorporated
• Inventors: Holloway, Lee Hahn, Prince, Matthew Browning, Tourne, Matthieu Philippe Franois
• Primary Examiner(s): Nguyen, Dustin
• Application Number:
  US12/939,923
• Publication Number:
  US 20120116896A1
• Time in Patent Office: 2,049 Days
• Field of Search: 709/206, 709/214, 709/226, 709/238, 709/224, 709/246, 709/218, 709/225, 709/229, 709/245, 709/219, 705/14, 707/999.01, 726/3, 715/234
• US Class Current: 1/1
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 40/14 Tree-structured documents parsing G06F40/205; validation G06F40/226
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: H04L 47/745 Reaction in network
• CPC Class Codes: H04L 51/42 Mailbox-related aspects, e.g. synchronisation of mailboxes
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 61/59 using proxies for addressing
• CPC Class Codes: H04L 63/0236 Filtering by address, protocol, port number or service, e.g. IP-address or URL
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 63/0254 Stateful filtering
• CPC Class Codes: H04L 63/0281 Proxies
• CPC Class Codes: H04L 63/083 : using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04L 63/0861 : using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 63/102 : Entity profiles
• CPC Class Codes: H04L 63/126 : the source of the received data
• CPC Class Codes: H04L 63/1416 : Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1433 : Vulnerability analysis
• CPC Class Codes: H04L 63/1441 : Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/1458 : Denial of Service
• CPC Class Codes: H04L 63/1466 : Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/146 : Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
• CPC Class Codes: H04L 67/56 : Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/561 : Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04L 67/568 : Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/40 : for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A proxy server receives from a client device a request for a network resource that is hosted at an origin server for a domain. The request is received at the proxy server as a result of a DNS request for the domain resolving to the proxy server. The origin server is one of multiple origin servers that belong to different domains that resolve to the proxy server and are owned by different entities. The proxy server retrieves the requested network resource. The proxy server determines that the requested resource is an HTML page, automatically modifies the HTML page, and transmits the modified HTML page to the client device.

Claims

1. A method in a proxy server to modify Internet responses, comprising:

receiving, from a client device, a request for a network resource that is hosted at an origin server for a domain, wherein the request is received at the proxy server as a result of a DNS (Domain Name System) request for the domain returning an IP address of the proxy server instead of an IP address of the origin server, wherein the origin server is one of a plurality of origin servers that belong to different domains in which DNS requests for each of those different domains return the same IP address of the proxy server instead of IP addresses of the respective origin servers, and wherein the pluarality of origin servers are owned by different entities;

retrieving the requested network resource;

determining that the requested network resource is an HTML (HyperText Markup Language) page;

automatically modifying the HTML page including automatically adding one or more scripts that, when executed by the clients device, performs one or more of adding content to the HTML page and modifying content included in the HTML page; and

transmitting the modified HTML page to the client device.

2. The method of claim 1, wherein retrieving the requested network resource includes performing the following:

determining whether the requested network resource is available in cache;

responsive to a determination that the requested network resource is not available in cache, performing the following;

transmitting the request to the origin server, and

receiving the network resource from the origin server;

responsive to a determination that the requested network resource is available in cache, accessing the cache to retrieve the requested network resource.

3. The method of claim 1, wherein adding the one or more scripts includes adding the one or more scripts includes adding to the HTML page one or more statistics scripts that include one or more of a tracking user statistics script and a tracking page performance statistics script.

4. The method of claim 1, wherein automatically modifying the HTML page includes performing the following:

adding to the HTML page one or more of;

a trap email address that is an email address that is not used for any real email and is uniquely identifiable for the IP address associated with the request, wherein the trap email address is added such that it is hidden from a rendered view of the HTML page while being capable of being read and used by bots that scan a source of the HTML page, and wherein receipt of an email message at the trap email address is an indication that a visitor associated with the request is a bot; and

a trap form that is a form that allows submission of information that is not used for any real communication and is uniquely identifiable for the IP address associated with the request, wherein the trap form is added such that it is hidden from a rendered view of the HTML page while being capable of being read and used by bots that scan a source of HTML page, and wherein receipt of submitted information through the trap form is an indication that the visitor is a bot.

5. The method of claim 1, wherein automatically modifying the HTML page includes performing the following:

scanning the HTML page for one or more advertisement token; and

responsive to a determination that the HTML page includes an advertisement token, performing the following;

determining a size and shape of the advertisement corresponding to the advertisement token; and

modifying the HTML page to cause a replacement advertisement of a similar size and shape to replace the advertisement corresponding to the advertisement token.

6. The method of claim 5, wherein the replacement advertisement is chosen based on one or more characteristics associated with the request including one or more of:

geographic location based on the IP address of the request, operating system type, date of the request, time of the request, and the domain.

7. The method of claim 5, wherein automatically modifying the HTML page to cause the replacement advertisement to replace the advertisement corresponding to the advertisement token includes creating a reference to the replacement advertisement that is stored on an external server.

8. The method of claim 5, wherein automatically modifying the HTML page to cause the replacement advertisement to replace the advertisement corresponding to the advertisement token includes embedding raw byte data of the replacement advertisement directly in the HTML page such that when loaded by a client network application of the client device, the replacement advertisement will be displayed on a rendered view of the HTML page without querying an external website for the advertisement.

9. A proxy server to modify Internet responses, comprising:

a memory to store instructions;

a processor coupled with the memory to process the stored instructions to;

receive, from a client device, a request for a network resource that is hosted at an origin server for a domain, wherein the request is received at the proxy server as a result of a DNS (Domain Name System) request for the domain returning an IP address of the proxy server instead of an IP address of the origin server, wherein the origin server is one of a plurality of origin servers that belong to different domains in which DNS requests for each of those different domains return the same IP address of the proxy server instead of IP addresses of the respective origin servers, and wherein the plurality of origin servers are owned by different entities;

retrieve the requested network resource;

determine that the requested network resource is an HTML (HyperText Markup Language) page;

automatically modify the HTML page including automatically add one or more scripts that, when executed by the client device, performs one or more of adding content to the HTML page and modifying content included in the HTML page; and

transmit the modified HTML page to the client device.

10. The proxy server of claim 9, wherein retrieve the requested network resource includes the processor to process the stored instructions to:

determine whether the requested network resource is available in cache;

responsive to a determination that the requested network resource is not available in cache, perform the following;

transmit the request to the origin server, and

receive the network resource from the origin server;

responsive to a determination that the requested network resource is available in cache, access the cache to retrieve the requested network resource.

11. The proxy server of claim 9, wherein the one or more scripts added to the HTML page include one or more of a tracking user statistics script and a tracking page performance statistics script.

12. The proxy server of claim 9, wherein automatically modify the HTML page includes the processor to process the stored instructions to perform the following:

add to the HTML page one or more of;

a trap email address that is an email address that is not used for any real email and is uniquely identifiable for the IP address associated with the request, wherein the trap email address is to be added such that it is hidden from a rendered view of the HTML page while being capable of being read and used by bots that scan a source of the HTML page, and wherein receipt of an email message at the trap email address is an indication that a visitor associated with the request is a bot; and

a trap form that is a form that allows submission of information that is not used for any real communication and is uniquely identifiable for the IP address associated with the request, wherein the trap form is to be added such that it is hidden from a rendered view of the HTML page while being capable of being read and used by bots that scan a source of HTML page, and wherein receipt of submitted information through the trap form is an indication that the visitor is a bot.

13. The proxy server of claim 9, wherein automatically modify the HTML page further includes the processor to process the stored instructions to perform the following:

scan the HTML page for one or more advertisement tokens; and

responsive to a determination that the HTML page includes an advertisement token, perform the following;

determine a size and shape of the advertisement corresponding to the advertisement token; and

modify the HTML page to cause a replacement advertisement of a similar size and shape to replace the advertisement corresponding to the advertisement token.

14. The proxy server of claim 13, wherein replacement of the advertisement is chosen based on one or more characteristics associated with the request including one or more of:

geographic location based on the IP address of the request, operating system type, date of the request, time of the request, and the domain.

15. The proxy server of claim 13, wherein modification of the HTML page to cause the replacement of the advertisement includes the processor to process the stored instructions to create a reference to the replacement advertisement that is stored on an external server.

16. The proxy server of claim 13, wherein modification of the HTML page to cause the replacement of the advertisement includes the processor to process the stored instructions to embed raw byte data of the replacement advertisement directly in the HTML page such that when loaded by a client network application of the client device, the replacement advertisement will be displayed on a rendered view of the HTML page without querying an external website for the advertisement.

17. A non-transitory machine-readable storage medium that provides instructions that, when executed by a processor of a proxy server, cause said processor to perform operations comprising:

receiving, from a client device, a request for a network resource that is hosted at an origin server for a domain, wherein the request is received at the proxy server as a result of a DNS (Domain Name System) request for the domain returning an IP address of the proxy server instead of an IP address of the origin server, wherein the origin server is one of a plurality of origin servers that belong to different domains in which DNS requests for each of those different domains return the same IP address of the proxy server instead of IP addresses of the respective origin servers, and wherein the plurality of origin servers are owned by different entities;

retrieving the requested network resource;

determining that the requested network resource is an HTML (HyperText Markup Language) page;

automatically modifying the HTML page including automatically adding one or more scripts that, when executed by the client device, performs one or more of adding content to the HTML page and modifying content included in the HTML page; and

transmitting the modified HTML page to the client device.

18. The non-transitory machine-readable storage medium of claim 17, wherein retrieving the requested network resource includes performing the following:

determining whether the requested network resource is available in cache;

responsive to a determination that the requested network resource is not available in cache, performing the following;

transmitting the request to the origin server, and receiving the network resource from the origin server;

responsive to a determination that the requested network resource is available in cache, accessing the cache to retrieve the requested network resource.

19. The non-transitory machine-readable storage medium of claim 17, wherein adding the one or more scripts includes adding to the HTML page one or more statistics scripts that include one or more of a tracking user statistics script and a tracking page performance statistics script.

20. The non-transitory machine-readable storage medium of claim 17, wherein automatically modifying the HTML page includes performing the following:

adding to the HTML page one or more of;

a trap email address that is an email address that is not used for any real email and is uniquely identifiable for the IP address associated with the request, wherein the trap email address is added such that it is hidden from a rendered view of the HTML page while being capable of being read and used by bots that scan a source of the HTML page, and wherein receipt of an email message at the trap email address is an indication that a visitor associated with the request is a bot; and

a trap form that is a form that allows submission of information that is not used for any real communication and is uniquely identifiable for the IP address associated with the request, wherein the trap form is added such that it is hidden from a rendered view of the HTML page while being capable of being read and used by bots that scan a source of HTML page, and wherein receipt of submitted information through the trap form is an indication that the visitor is a bot.

21. The non-transitory machine-readable storage medium of claim 17, wherein automatically modifying the HTML page further includes performing the following:

scanning the HTML page for one or more advertisement tokens; and

responsive to a determination that the HTML page includes an advertisement token, performing the following;

determining a size and shape of the advertisement corresponding to the advertisement token; and

modifying the HTML page to cause a replacement advertisement of a similar size and shape to replace the advertisement corresponding to the advertisement token.

22. The non-transitory machine-readable storage medium of claim 21, wherein the replacement advertisement is chosen based on one or more characteristics associated with the request including one or more of:

geographic location based on the IP address of the request, operating system type, date of the request, time of the request, and the domain.

23. The non-transitory machine-readable storage medium of claim 21, wherein automatically modifying the HTML page to cause the replacement advertisement to replace the advertisement corresponding to the advertisement token includes creating a reference to the replacement advertisement that is stored on an external server.

24. The non-transitory machine-readable storage medium of claim 21, wherein automatically modifying the HTML page to cause the replacement advertisement to replace the advertisement corresponding to the advertisement token includes embedding raw byte data of the replacement advertisement directly in the HTML page such that when loaded by a client network application of the client device, the replacement advertisement will be displayed on a rendered view of the HTML page without querying an external website for the advertisement.