Web reputation scoring

• US 8,561,167 B2
• Filed: 01/24/2007
• Issued: 10/15/2013
• Est. Priority Date: 03/08/2002
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Alperovitch, Dmitri, Foote-Lennox, Tomo, Greve, Paula, Judge, Paul, Krasser, Sven, Lange, Tim, Schneck, Phyllis Adele, Stecher, Martin, Tang, Yuchun, Zdziarski, Jonathan Alexander
• Primary Examiner(s): HERZOG, MADHURI R
• Application Number:
  US11/626,470
• Publication Number:
  US 20070130350A1
• Time in Patent Office: 2,456 Days
• Field of Search: 726/2, 726 22- 25, 726 11- 13, 709204-207, 709217-229, 379/201.01, 379/210.02, 379/210.03, 713151-154
• US Class Current: 726/13
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/168 above the transport layer

Abstract

Methods and systems for operation upon one or more data processors for assigning reputation to web-based entities based upon previously collected data.

Claims

1. A computer implemented method operable to assign a reputation to a web-based entity associated with a hypertext transfer protocol communication, comprising:

receiving, at a local reputation engine, a hypertext transfer protocol communication at an edge protection device;

identifying, at the local reputation engine, an entity associated with the received hypertext transfer protocol communication;

querying, from the local reputation engine, a global reputation server using a query for a reputation indicator associated with the entity;

receiving, at a local reputation engine, the reputation indicator from the global reputation server; and

taking an action with respect to the hypertext transfer protocol communication based upon the received reputation indicator associated with the entity;

wherein;

a reputation of the entity is based upon previous communications received from the entity, the previous communications being previous communications of two or more of the following communication types;

a hypertext transfer protocol communication, an instant message, a file transfer protocol communication, simple object access protocol messages, real-time transport protocol packages, a short message service communication, multimedia message service communication, or a voice over internet protocol communication; and

wherein the reputation is determined from;

collecting, at each of a plurality of other local reputation engines, the previous communications that are respectively received by the local reputation engine;

determining, at each of the plurality of other local reputation engines, identifiers for each of the previous communications, each identifier for each previous communication identifying a sending entity associated with the previous communication, and sending entities include the entity associated with the received hypertext transfer protocol communication;

determining, at each of the plurality of other local reputation engines, attributes for each of the previous communications, the attributes indicative of a reputation of the sending entity associated with a previous communication;

determining, at each of the plurality of other local reputation engines, a local reputation of the sending entities from the identifiers and the attributes from the previous communications, wherein at least some of the reputations are based on the similarity of attributes associated with two or more identifiers; and

aggregating the local reputations at the global reputation server to generate respective reputation indicators, each local reputation produced by an associated local reputation engine, and the local reputation weighted based on a confidence value associated with the respective local reputation engine, wherein the confidence value is based at least in part on historical performance data for the local reputation engine, the historical performance data comprising statistics based at least in part on numbers of entities incorrectly classified by the local reputation engine.

2. The method of claim 1, wherein the entity is a web entity comprising a destination universal resource locator, domain or IP address.

3. The method of claim 1, wherein the reputation of the entity is further based upon public or private network information available about the entity comprising ownership or hosting information.

4. The method of claim 1, wherein the action is to discard the communication and notify an enterprise network user associated with the hypertext transfer protocol communication.

5. The method of claim 1, wherein the global reputation server generates the reputation indicator based upon an aggregation of reputable criteria associated with the entity and non-reputable criteria associated with the entity.

6. The method of claim 5, wherein the reputation indicator is a reputation vector which indicates reputation based upon a plurality of different criteria.

7. The method of claim 6, further comprising examining the reputation vector to determine whether a policy associated with an enterprise network protected by the edge protection device allows communication with the entity based upon its reputation vector.

8. The method of claim 1, wherein the global reputation server reputation engine is operable to provide a plurality of network security devices with one or more lists of reputations, the one or more lists of reputations comprising reputation information associated with a universe of entities that have associated reputations.

9. The method of claim 1, wherein the global reputation server is a reputation server operable to provide a plurality of edge protection devices with reputation information.

10. The method of claim 9, wherein the global reputation server reputation engine is operable to store a global reputation indicator and to bias the global reputation indicator with a local bias prior to outputting the reputation indicator.

11. The method of claim 1, wherein the reputation indicator comprises a reputation vector, the reputation vector comprising a multi-dimensional classification of the entity.

12. The method of claim 11, wherein the multi-dimensional classification comprises classification of the message in two or more of a porn category, a news category, a computer category, a secure category, a phishing category, a spyware category, a virus category, or an attack category.

13. The method of claim 11, wherein the reputation indicator further comprises a confidence associated with each of the multi-dimensional classifications of the entity.

14. The method of claim 1, further comprising detecting randomization of a universal resource locator.

15. The method of claim 14, wherein the randomization of a universal resource locator is determined by generating a hash of the universal resource locator and comparing the hash to previously identified non-reputable universal resource locators.

16. The method of claim 14, wherein the randomization of a universal resource locator is determined by generating fingerprints of a plurality of portions of the universal resource locator and comparing the hash one or more of the fingerprints to previously identified non-reputable universal resource locators.

34. The method of claim 1, wherein the confidence value is further based upon user feedback indicating a performance of the local reputation engine with respect to actions on communications.

17. A web reputation system implemented in one or more computer devices, the web reputation system operable to receive a web communication and to assign a reputation to an entity associated with the communication, the system comprising:

a communications interface device operable to receive a web communication;

computer memory operable to store the web communication;

a communication analyzer operable to analyze the web communication to determine an entity associated with the web communication; and

a local reputation engine operable to;

query a global reputation server using a query for a reputation indicator associated with the entity based upon previously collected data associated with the entity;

receive the reputation indicator from the global reputation server; and

determine whether the web communication is to be communicated to a recipient;

wherein;

a reputation of the entity is based upon previous communications received from the entity, the previous communications being previous communications of two or more of the following communication types;

a hypertext transfer protocol communication, an instant message, a file transfer protocol communication, simple object access protocol messages, real-time transport protocol packages, a short message service communication, multimedia message service communication, or a voice over internet protocol communication; and

wherein the reputation is determined from;

collecting, at each of a plurality of other local reputation engines, the previous communications that are respectively received by the local reputation engine;

determining, at each of the plurality of other local reputation engines, identifiers for each of the previous communications, each identifier for each previous communication identifying a sending entity associated with the previous communication, and sending entities include the entity associated with the received web communication;

determining, at each of the plurality of other local reputation engines, attributes for each of the previous communications, the attributes indicative of a reputation of the sending entity associated with a previous communication;

determining, at each of the plurality of other local reputation engines, a local reputations of the sending entities from the identifiers and the attributes from the previous communications, wherein at least some of the reputations are based on the similarity of attributes associated with two or more identifiers; and

aggregating the local reputations at the global reputation server to generate respective reputation indicators, each local reputation produced by an associated local reputation engine, and the local reputation weighted based on a confidence value associated with the respective local reputation engine, wherein the confidence value is based at least in part on historical performance data for the local reputation engine, the historical performance data comprising statistics based at least in part on numbers of entities incorrectly classified by the local reputation engine.

18. The system of claim 17, wherein the local reputation engine is operable to notify an enterprise network user associated with the hypertext transfer protocol communication in the event that the communication is not transmitted to the recipient.

19. The system of claim 17, wherein the global reputation server determines the reputation indicator based upon an aggregation of reputable criteria associated with the entity and non-reputable criteria associated with the entity.

20. The system of claim 19, wherein the reputation indicator is a reputation vector which indicates reputation based upon a plurality of different criteria.

21. The system of claim 20, further comprising examining the reputation vector to determine whether a policy associated with an enterprise network protected by an edge protection device allows communication with the entity based upon its reputation vector.

22. The system of claim 17, wherein the global reputation server is a reputation server operable to provide a plurality of edge protection devices with reputation information.

23. The system of claim 22, wherein the global reputation server is operable to store a global reputation indicator and to bias the global reputation indicator with a local bias prior to outputting the reputation indicator.

24. The system of claim 17, further comprising an interrogation engine operable to perform a plurality of tests on the communication and to determine a profile associated with the web communication.

25. The system of claim 24, wherein the local reputation engine is operable to determine whether to forward the web communication based upon the profile associated with the web communication.

26. The system of claim 25, wherein the local reputation engine is operable to use the profile to update reputation information associated with the entity.

27. The system of claim of claim 17, wherein the reputation comprises a reputation vector, the reputation vector comprising a multi-dimensional classification of the entity.

28. The system of claim 27, wherein the multi-dimensional classification comprises classification of the message in two or more of a porn category, a news category, a computer category, a secure category, a phishing category, a spyware category, a virus category, or an attack category.

29. The system of claim 27, wherein the reputation further comprises a confidence associated with each of the multi-dimensional classifications of the entity.

30. The system of claim 17, further comprising detecting randomization of a universal resource locator.

31. The system of claim 30, wherein the randomization of a universal resource locator is determined by generating a hash of the universal resource locator and comparing the hash to previously identified non-reputable universal resource locators.

32. The system of claim 30, wherein the randomization of a universal resource locator is determined by generating fingerprints of a plurality of portions of the universal resource locator and comparing one or more of the fingerprints to previously identified non-reputable universal resource locators.

33. One or more non-transitory computer readable media having software program code operable to assign a reputation to a messaging entity associated with a received communication, comprising:

receiving a hypertext transfer protocol communication at an edge protection device;

identifying, at the edge protection device, an entity associated with the received hypertext transfer protocol communication;

querying, from the edge protection device, a global reputation server using a query for a reputation indicator associated with the entity;

receiving, at the edge protection device, the reputation indicator from the global reputation server; and

taking an action with respect to the hypertext transfer protocol communication based upon the received reputation indicator associated with the entity;

wherein;

a reputation of the entity is based upon previous communications received from the entity, the previous communications being previous communications of two or more of the following communication types;

a hypertext transfer protocol communication, an instant message, a file transfer protocol communication, simple object access protocol messages, eal-time transport protocol packages, a short message service communication, multimedia message service communication, or a voice over internet protocol communication; and

wherein the reputation indicator is determined from;

collecting, at each of a plurality of other edge protection devices, the previous communications that are respectively received by the edge protection device;

determining, at each of the plurality of other edge protection devices, identifiers for each of the previous communications, each identifier for each previous communication identifying a sending entity associated with the previous communication, and sending entities include the entity associated with the received hypertext transfer protocol communication;

determining, at each of the plurality of other edge protection devices, attributes for each of the previous communications, the attributes indicative of a reputation of the sending entity associated with a previous communication;

determining, at each of the plurality of other edge protection devices, a local reputation of the sending entities from the identifiers and the attributes from the previous communications, wherein at least some of the reputations are based on the similarity of attributes associated with two or more identifiers; and

aggregating the local reputations at the global reputation server to generate respective reputation indicators, each local reputation produced by an associated edge protection device, and the local reputation weighted based on a confidence value associated with the respective edge protection device, wherein the confidence value is based at least in part on historical performance data for the edge protection device, the historical performance data comprising statistics based at least in part on numbers of entities incorrectly classified by the edge protection device.

35. A computer implemented method, comprising:

receiving, at a local reputation engine, a hypertext transfer protocol communication at an edge protection device;

identifying, at the local reputation engine, an entity associated with the received hypertext transfer protocol communication;

querying, from the local reputation engine, a global reputation server using a query for a reputation indicator associated with the entity;

receiving, at a local reputation engine, the reputation indicator from the global reputation server; and

taking an action with respect to the hypertext transfer protocol communication based upon the received reputation indicator associated with the entity;

wherein;

a reputation of the entity is based upon previous communications received from the entity, the previous communications being previous communications of two or more of the following communication types;

a hypertext transfer protocol communication, an instant message, a file transfer protocol communication, simple object access protocol messages, real-time transport protocol packages, a short message service communication, multimedia message service communication, or a voice over internet protocol communication; and

wherein the reputation is determined from;

collecting, at each of a plurality of other local reputation engines, the previous communications that are respectively received by the local reputation engine;

determining, at each of the plurality of other local reputation engines, identifiers for each of the previous communications, each identifier for each previous communication identifying a sending entity associated with the previous communication, and sending entities include the entity associated with the received hypertext transfer protocol communication;

determining, at each of the plurality of other local reputation engines, attributes for each of the previous communications, the attributes indicative of a reputation of the sending entity associated with a previous communication;

determining, at each of the plurality of other local reputation engines, a local reputation of the sending entities from the identifiers and the attributes from the previous communications, wherein at least some of the reputations are based on the similarity of attributes associated with two or more identifiers; and

aggregating the local reputations at the global reputation server to generate respective reputation indicators, each local reputation produced by an associated local reputation engine, and the local reputation weighted based on a confidence value associated with the respective local reputation engine, wherein the confidence value is based on user feedback indicating a performance of the local reputation engine with respect to actions on communications.