System and method for analyzing web content

• US 9,003,524 B2
• Filed: 12/23/2013
• Issued: 04/07/2015
• Est. Priority Date: 07/10/2006
• Status: Active Grant

• Current Assignee: Forcepoint LLC
• Original Assignee: Forcepoint LLC
• Inventors: Baddour, Victor L., Chenette, Stephan, Hubbard, Dan, Mesdaq, Ali A., Verenini, Nicholas J
• Primary Examiner(s): Truong, Thanhnga B
• Assistant Examiner(s): PLECHA, THADDEUS J
• Application Number:
  US14/139,597
• Publication Number:
  US 20140115699A1
• Time in Patent Office: 470 Days
• Field of Search: 726/22
• US Class Current: 726/22
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: G06F 2221/034 Test or assess a computer or a system
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/14 for detecting or protecting against malicious traffic
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/168 above the transport layer

Abstract

A system and computer based method are provided for identifying active content in websites on a network. One embodiment includes a computer based method of classifying web content. The method receives content of a web page, and determines a first property associated with the content, the first property including static content. The method executes active content associated with the webpage, and determines a second property associated with the content based at least in part on the executing, the second property including the active content. The method also evaluates a logical expression relating the first property and the second property, and associates the web page with a category based on a result of the evaluation. The evaluation of the logical expression at least in part evaluates whether a constant value matches at least a portion of the content of the web page.

Claims

1. A method of classifying web content, implemented on one or more computer processors, the method comprising:

using at least one of the processors, receiving content of a web page;

using at least one of the processors, determining a first property associated with the content of the web page, the first property comprising static content associated with the web page;

using at least one of the processors, executing active content associated with the webpage;

using at least one of the processors, determining a second property associated with the content of the web page based at least in part on the executing, the second property comprising the active content;

using at least one of the processors, evaluating a logical expression relating the first property and the second property; and

using at least one of the processors, associating the web page with a category based on a result of the evaluation,

wherein the evaluation of the logical expression at least in part evaluates whether a constant value matches at least a portion of the content of the web page.

2. The method of claim 1, wherein executing the active content comprises executing the active content in a sandbox environment.

3. The method of claim 1, wherein determining the first property associated with content of the web page comprises identifying a keyword in the content of the web page.

4. The method of claim 1, wherein the first property is associated with a regular expression matching at least a portion of the content of the web page.

5. The method of claim 1, wherein the first property includes an indication of a number of occurrences of a string within the web page.

6. The method of claim 1, wherein evaluating the logical expression comprises relating each of the first and second properties to corresponding constant values, and wherein the method further comprises assigning the category to the web page if the evaluated logical expression corresponds to a predetermined value.

7. The method of claim 6, wherein relating the first property to its corresponding constant value comprises evaluating a regular expression based on a content of the web page, and relating the result of the evaluation to the corresponding constant value.

8. The method of claim 1, further comprising:

evaluating a plurality of logical expressions based on the content of the web page, wherein each of the logical expressions relate the first property and the second property, wherein each of the plurality of logical expressions corresponds to a web page category; and

associating the web page with a logical expression'"'"'s corresponding category if the evaluated logical expression corresponds to a predetermined value.

9. A system for classifying web content, the system comprising:

one or more hardware processors configured, individually or in combination, to;

receive content of a web page;

determine a first property associated with the content of the web page, the first property comprising static content associated with the web page;

determine a second property associated with the content of web page based at least in part by executing active content associated with the webpage, the second property comprising the active content;

evaluate a logical expression relating the first property and the second property; and

associate the web page with a category based on a result of the evaluation,

wherein the evaluation at least in part evaluates whether a constant value matches at least a portion of the content of the web page.

10. The system of claim 9, wherein the one or more processors are further configured, individually or in combination, to execute the active content in a sandbox environment.

11. The system of claim 9, wherein the one or more processors are further configured, individually or in combination, to determine the first property of the web page at least in part by identifying a keyword in the content of the web page.

12. The system of claim 9, wherein the first property is associated with a regular expression matching at least a portion of the content of the web page.

13. The system of claim 9, wherein the first property includes an indication of a number of occurrences of a string within the web page.

14. The system of claim 9, wherein evaluating the logical expression comprises relating each of the first and second properties to corresponding constant values, and wherein the one or more processor are further configured to assign the category to the web page if the evaluated logical expression corresponds to a predetermined value.

15. The system of claim 14, wherein relating the first property to its corresponding constant value comprises evaluating a regular expression on a content of the web page, and relating the result of the evaluation to the corresponding constant value.

16. The system of claim 9, wherein the one or more processors are further configured, individually or in combination, to:

evaluate a plurality of logical expressions based on the content of the web page, wherein each of the logical expressions relates the first property and the second property, and wherein each of the plurality of logical expressions corresponds to a web page category; and

associate the web page with a logical expression'"'"'s corresponding category if the evaluated logical expression corresponds to a predetermined value.

17. An apparatus for classifying web content, implemented on one or more computer processors, comprising:

means for determining a first property associated with content of a web page, the first property comprising static content associated with the web page;

means for executing active content associated with the webpage;

means for determining a second property associated with the content of the web page based at least in part on the executing, the second property comprising the active content;

means for evaluating a logical expression relating the first property and the second property; and

means for associating the web page with a category based on a result of the evaluation,

wherein the means for evaluating a logical expression relating the first'"'"' property and the second property is configured to evaluate whether a constant value matches at least a portion of the content of the web page.

18. The apparatus of claim 17, wherein the means for associating the web page with a category based on a result of the evaluation comprises one or more hardware processors.