Detecting computer security threats in electronic documents based on structure

• US 9,124,622 B1
• Filed: 11/07/2014
• Issued: 09/01/2015
• Est. Priority Date: 11/07/2014
• Status: Active Grant

• Current Assignee: CloudFlare Incorporated
• Original Assignee: Area 1 Security, Inc.
• Inventors: Falkowitz, Oren, Syme, Philip
• Primary Examiner(s): Smithers, Matthew
• Application Number:
  US14/536,534
• Time in Patent Office: 298 Days
• Field of Search: 726/22
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0275 Auctions
• CPC Class Codes: H04L 63/02 for separating internal from external traffic, e.g. firewalls
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

In an embodiment, a data processing method providing an improvement in computer security comprises selecting, from a queue identifying a plurality of web pages, a particular web page to retrieve from one of a plurality of internet sources; causing retrieving a copy of the particular web page from a particular internet source; determining a hierarchical structure of the particular web page; based upon a hierarchical structure of the particular web page and without consideration of content of the particular web page, identifying one or more features, of links in the particular web page or files referenced in the particular web page, that indicate one or more security threats; determining a reputation score for the particular web page; determining a specified remediation measure, based upon the reputation score, to remediate a security threat that is identified in the particular web page; providing the specified remediation measure to one or more of a compromised computer, a sensor computer and an enterprise computer.

Claims

1. A data processing method providing an improvement in computer security, comprising:

selecting, from a queue identifying a plurality of web pages, a particular web page to retrieve from one of a plurality of internet sources;

causing retrieving a copy of the particular web page from a particular internet source;

determining a hierarchical document object model (DOM) tree structure of the particular web page;

based upon the hierarchical DOM tree structure of the particular web page and independent of consideration of content of the particular web page, identifying one or more features, of links in the particular web page or files referenced in the particular web page, that indicate one or more security threats;

determining a reputation score for the particular web page based on the one or more features that indicate the one or more security threats;

determining a specified remediation measure, based upon the reputation score, to remediate a security threat that is identified in the particular web page;

providing the specified remediation measure to one or more of a compromised computer, a sensor computer and an enterprise computer.

2. The method of claim 1, further comprising reviewing each link in the particular web page, computing a link score based upon a number of redirection paths that are associated with that link, and determining the reputation score at least in part based upon the link score.

3. The method of claim 2, further comprising refraining from using a particular link to reach another web page referenced in the link when the link score is greater than a specified threshold that is associated with non-traversable links.

4. The method of claim 1, further comprising inspecting each of the files referenced in the particular web page, determining a status value for each of the files, and determining the reputation score at least in part based upon the status value.

5. The method of claim 1, further comprising determining the reputation score at least in part based upon combining the reputation score with one or more enrichment data sources.

6. The method of claim 1, further comprising determining the reputation score at least in part based upon combining the reputation score with one or more of geo-location data and one or more other enrichment sources.

7. The method of claim 1, further comprising performing the selecting based upon advertising exchange network bid data.

8. The method of claim 1, further comprising performing the selecting by:

extracting, from a first web page, web page metadata comprising one or more of;

a DNS name of a domain of a URL of the web page;

an IP address of a web server that serves the first web page;

a URL of the first web page;

a source page containing a link that was used to retrieve the first web page;

based upon the web page metadata, determining a plurality of second web pages to fetch and an order of fetching the plurality of second web pages.

9. The method of claim 1, further comprising determining the reputation score for the particular web page, determining the specified remediation measure, and providing the specified remediation measure only when the hierarchical DOM tree structure of the particular web page has not changed since a last inspection of the same particular web page.

10. The method of claim 1, wherein the steps of the method are performed using inspecting logic in a security control computer that is coupled through one or more networks to a sensor computer that is co-located with and coupled to a compromised computer, and further comprising providing the specified remediation measure only to the sensor computer.

11. The method of claim 1, wherein the steps of the method are performed using inspecting logic in a security control computer that is coupled through one or more networks to a sensor computer that is co-located with and coupled to a compromised computer that is coupled to a firewall that is configured to control ingress of packets to the compromised computer, and further comprising providing the specified remediation measure only to the firewall.

12. The method of claim 1, wherein the queue is coupled to inspecting logic that is configured to inspecting logic that is configured to store the plurality of web pages in the queue based upon input from one or more of:

advertising exchange network bid data;

global user browsing data indicating which web pages are the subject of browsing by a plurality of geographically distributed users;

feedback from analysis of previous web pages by the inspecting logic;

a user query;

scheduling input.

13. A data processing system providing an improvement in computer security, comprising:

a plurality of sensor computers, each of which is coupled to different one among a plurality of compromised computers in geographically distributed locations, each of the compromised computers comprising at least one malware item that is configured to direct unauthorized network activity toward one or more enterprise networks or enterprise computers, wherein the compromised computers are logically between one or more attacker computers and the one or more enterprise networks or enterprise computers;

a security control computer that is coupled to the sensor computers;

one or more non-transitory data storage media in the security control computer storing security logic comprising one or more sequences of instructions which when executed cause the security control computer to perform;

selecting, from a queue identifying a plurality of web pages, a particular web page to retrieve from one of a plurality of internet sources;

causing retrieving a copy of the particular web page from a particular internet source;

determining a hierarchical DOM tree structure of the particular web page;

based upon a hierarchical DOM tree structure of the particular web page and independent of consideration of content of the particular web page, identifying one or more features, of links in the particular web page or files referenced in the particular web page, that indicate one or more security threats;

determining a reputation score for the particular web page based on the one or more features that indicate the one or more security threats;

determining a specified remediation measure, based upon the reputation score, to remediate a security threat that is identified in the particular web page;

providing the specified remediation measure to one or more of a compromised computer, a sensor computer and an enterprise computer.

14. The system of claim 13, the security logic further comprising instructions which when executed cause reviewing each link in the particular web page, computing a link score based upon a number of redirection paths that are associated with that link, and determining the reputation score at least in part based upon the link score.

15. The system of claim 14, the security logic further comprising instructions which when executed cause refraining from using a particular link to reach another web page referenced in the link when the link score is greater than a specified threshold that is associated with non-traversable links.

16. The system of claim 13, the security logic further comprising instructions which when executed cause inspecting each of the files referenced in the particular web page, determining a status value for each of the files, and determining the reputation score at least in part based upon the status value.

17. The system of claim 13, the security logic further comprising instructions which when executed cause performing the selecting based upon advertising exchange network bid data.

18. The system of claim 13, the security logic further comprising instructions which when executed cause performing the selecting by:

extracting, from a first web page, web page metadata comprising one or more of;

a DNS name of a domain of a URL of the web page;

an IP address of a web server that serves the first web page;

a URL of the first web page;

a source page containing a link that was used to retrieve the first web page;

based upon the web page metadata, determining a plurality of second web pages to fetch and an order of fetching the plurality of second web pages.

19. The system of claim 13, the security logic further comprising instructions which when executed cause determining the reputation score for the particular web page, determining the specified remediation measure, and providing the specified remediation measure only when the hierarchical DOM tree structure of the particular web page has not changed since a last inspection of the same particular web page.

20. The system of claim 13, wherein the queue is coupled to inspecting logic that is configured to store the plurality of web pages in the queue based upon input from one or more of:

advertising exchange network bid data;

global user browsing data indicating which web pages are the subject of browsing by a plurality of geographically distributed users;

feedback from analysis of previous web pages by the inspecting logic;

a user query;

scheduling input.