USER-CONTROLLABLE LEARNING OF POLICIES

• US 20100036779A1
• Filed: 05/16/2009
• Published: 02/11/2010
• Est. Priority Date: 05/16/2008
• Status: Active Grant

• Current Assignee: Proofpoint Incorporated
• Original Assignee: Carnegie Mellon University, Wombat Security Technologies, Inc (Proofpoint Incorporated)
• Inventors: Kelley, Patrick Gage, Sadeh-Koniecpol, Norman, Cranor, Lorrie Faith, Drielsma, Paul Hankes
• Granted Patent: US 8,423,483 B2
• Time in Patent Office: Days
• US Class Current: 706/11
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

Various embodiments are directed to a computer implemented method for updating a policy that is enforced by a computer program. In one embodiment, a computer communicates, to a user, data regarding one or more decisions made by the program over a period of time according to a policy. Each decision is made on the particular policy in force at the time the decision is made. Policy data for the policy is stored in a machine readable format. The user feedback data indicative of feedback by the user regarding the one or more decisions is stored. The computer identifies and ranks one or more potential variations to the policy based on a score of an objective function for each potential variation. The computer communicates, to the user, one or more suggested modifications based on the ranking of the one or more potential variations to the policy. The computer modifies the policy data based on one or more selections by the user in response to the transmission of the one or more suggested modifications.

Claims

1. A computer implemented method for updating a policy that is enforced by a computer program, the method comprising:

communicating, by a computer system to at least one user, data regarding one or more decisions made by the program over a period of time according to a policy, wherein each decision is made on the particular policy in force at the time the decision is made, and wherein policy data for the policy is stored in a machine readable format in a memory coupled to the computer system, wherein the computer system comprises at least one processor coupled to a memory;

storing in a machine readable format user feedback data indicative of feedback by the at least one user regarding the one or more decisions made by the program according to the policy;

identifying and ranking, by the computer system, one or more potential variations to the policy based on a score of an objective function, wherein the objective function comprises a plurality of function elements, wherein the plurality of function elements comprises a user satisfaction function element and at least one other function element that measures an extent to which the one or more potential variations are likely to be understandable by the at least one user;

communicating, by the computer system to the at least one user, one or more suggested modifications to the policy based on the ranking of the one or more potential variations to the policy, wherein the one or more suggested modifications can assist the one or more users refine the policy; and

modifying, by the computer system, the policy data stored in the memory based on one or more selections by the at least one user in response to the communication of the one or more suggested modifications.

2. The method of claim 1, wherein the plurality of function elements includes a function that measures the likelihood that the one or more policy variations will be understandable by the at least one user based on:

measures of the complexity of the policy variations under consideration; and

measures of the deviation between the policy variation under consideration and the policy currently in force.

3. The method of claim 2, wherein the plurality of function elements comprises a function that measures the diversity of the policy variations presented to the at least one user with the objective of presenting the at least one user with a meaningfully diversified set of policy variations.

4. The method of claim 1, wherein a selection made by the at least one user when presented with the one or more policy variations includes accepting, rejecting and modifying the one or more policy variations.

5. The method of claim 1, wherein modifying, by the computer system, the policy data stored in the memory can be initiated by the at least one user independently of any suggested modifications.

6. The method of claim 1, comprising identifying and ranking one or more potential variations to the policy comprising one or more neighboring policies.

7. The method of claim 1, comprising:

evaluating the one or more policy variation alone or in combination in accordance with the objective function; and

selecting the one or more policy variations in accordance with the score provided by the objective function.

8. The method of claim 1, comprising determining a user satisfaction metric based on the feedback by the at least one user.

9. The method of claim 2, comprising determining a deviation metric that measures by how much each of the one or more potential variations to a policy under consideration deviates from the policy currently in force, wherein the objective function comprises a deviation function element.

10. The method of claim 9, comprising determining the deviation metric in accordance with one or more sub-function elements, wherein the sub-function elements are selected from the group consisting of (i) a function based on the number of individual variation steps required to transform the policy currently in force into the potential policy variation under consideration, wherein variation steps are limited to variation steps that have been identified as easy to understand in a domain under consideration;

(ii) a function that captures the effective difference between the potential policy variation under consideration and the policy currently in force, when the policy specifies a finite number of decisions;

(iii) a function of user tolerance for deviation as collected through experiments with a representative set of users;

(iv) a function of user tolerance for deviation as measured based on historical data comprising user feedback; and

(v) a function based on the volumetric difference between the policy currently in force and the potential policy variation under consideration when represented as geometric shapes in a multidimensional space, and (vi) domain-specific sub-function elements.

11. The method of claim 2, comprising determining a complexity metric that measures how complex each of the one or more potential variations to a policy under consideration is, wherein the objective function comprises a complexity function element.

12. The method of claim 11, comprising determining the complexity metric in accordance with one or more sub-function elements, wherein the sub-function elements are selected from the group consisting of:

(i) a function of a number of rules comprising the policy;

(ii) a function of a complexity of the rules based on attributes comprising any one of conditions, options, and actions associated with each rule;

(iii) a function of complexity based on data collected through experiments with a representative set of users;

(iv) a function of complexity based on historical data comprising user feedback;

(v) a function based on patterns among the rules and patterns within individual rules, including entropy metrics; and

(vi) domain-specific sub-function elements.

13. The method of claim 3, comprising determining a diversity metric that measures how diverse a combination of the one or more potential variations to the policy under consideration is;

and evaluating the one or more suggested modifications are further evaluated according to a diversity function element.

14. The method of claim 13, comprising determining the diversity of a combination of policy variations in accordance with one or more sub-function elements, wherein the sub-function elements are selected from the group consisting of:

(i) a function based on the number and/or type of individual variation steps required to transform each of the one or more policy variations in the combination into any other policy variation in the combination;

(ii) a function based on the number and/or type of individual variation steps required to transform the policy currently in force into each of the policy modifications in a combination under consideration;

(iii) a function of the entropy of the policy modifications in a combination under consideration; and

(iv) domain-specific sub-function elements.

15. The method of claim 1, wherein the objective function comprises a general objective function comprising the user satisfaction metric function element and further comprising:

determining a deviation metric to measure by how much each of the one or more potential variations to a policy under consideration deviates from the policy currently in force in accordance with a deviation function element;

determining a complexity metric to measure how complex each of the one or more potential variations to a policy under consideration is, in accordance with a complexity function element; and

determining a diversity metric to measure how diverse the set of one or more potential variations to the policy under consideration is, in accordance with a diversity function element.

16. The method of claim 15, comprising determining the general objective function (GOF) according to the following combination:

GOF=user satisfaction metric+deviation metric+complexity metric+diversity metric.

17. The method of claim 1, wherein the objective function comprises a policy evaluation function comprising the user satisfaction function element to determine a user satisfaction metric based on the feedback by the at least one user, and further comprising a combination of any one of:

determining a deviation metric to determine by how much each of the one or more potential variations to a policy under consideration deviates from the policy currently in force, in accordance with a deviation function element;

determining a complexity metric to determine how complex each of the one or more potential variations to a policy under consideration is, in accordance with a complexity function element; and

determining a diversity metric to determine how diverse the one or more potential variations to the policy under consideration is, in accordance with a diversity function element.

18. The method of claim 1, comprising applying the suggested modifications subject to acceptance by the at least one user.

19. The method of claim 1, wherein the at least one user comprises any one of an end user, an administrator of the policy enforcing computer system, and a person associated with specifying and modifying the policy.

20. The method of claim 1, comprising receiving implicit feedback from the at least one user regarding the one or more decisions made by the program according to the policy.

21. The method of claim 20, comprising considering the feedback to be implicit feedback unless the at least one user provides explicit feedback.

22. The method of claim 21, comprising applying different weights to the implicit feedback by the user satisfaction function element according to whether the implicit feedback is determined positively or negatively in the absence of explicit feedback from the at least one user.

23. The method of claim 21, comprising determining implicit feedback in accordance with one or more variables selected from the group consisting of the relative recentness of the one or more decisions made by the program, what users provided the feedback, and the roles played by the users.

24. The method of claim 21, comprising combining the feedback data provided by two or more users.

25. The method of claim 1, wherein communication to the at least one user comprises presenting to the user the one or more suggested modifications through any one of a visual representation, a textual representation, an audible representation, and any other meaningful method of interaction with user.

26. The method of claim 1, comprising any one of accepting, rejecting, and modifying the one or more suggested modifications by the at least one user in response to the communication of the one or more suggested modifications.

27. The method of claim 1, comprising:

interacting with the policy by a plurality of users; and

reviewing one or more suggested modifications to the policy by at least one of the plurality of users;

selecting one or more of the one or more suggested modifications to the policy by the at least one of the plurality of users; and

providing feedback on the policy.

28. The method of claim 27, comprising selecting, by the plurality of users, the one or more suggested modifications to the policy based on a vote.

29. The method of claim 1, comprising selectively disclosing location information of the at least one user in accordance with the policy, wherein the policy is applied in the context of a location-sharing application.

30. The method of claim 1, comprising filtering by a firewall in response to incoming and outgoing packets on a network in accordance with the policy, wherein the policy is applied in the context of a network firewall.

31. The method of claim 1, comprising controlling behavior of a network in response to traffic on the network that pertains to specific applications in accordance with the policy, wherein the policy is applied in the context of an application level firewall.

32. The method of claim 1, comprising determining patterns of traffic on a network indicative of a possible intrusion, or a possible data leakage, or a possible regulatory violation, or a possible violation of corporate or departmental policy, in accordance with the policy, wherein the policy is applied in the context of a relevant detection system.

33. The method of claim 1, comprising selectively disclosing personal information including any one of a calendar, status information, pictures, movies, contextual information and any other relevant information users may want to selectively share with others in accordance with the policy, wherein the policy is applied in the context of social networking applications and may be either an individual policy, a policy applying to a group of users, or a policy applying to all users of a social networking application.

34. The method of claim 1, comprising routing packets on a network in accordance with the policy, wherein the policy is applied in the context of routing devices on the network.

35. The method of claim 1, comprising controlling routers, firewalls, and access points in accordance with the policy, and controlling behavior, connectivity, and security of a home networking device in accordance with the policy, wherein the policy is as applied in the context of home networking devices.

36. The method of claim 1, comprising defining a model of the preferences, likes, and dislikes of the at least one user pertaining to products and services in accordance with the policy, wherein the policy is applied in the context of recommender systems for products and services.

37. The method of claim 1, comprising defining a set of patterns within computer programs that represent errors or inefficiencies and a set of proposed improvements to the errors or inefficiencies in accordance with the policy, wherein the policy is applied in the context of software development environments for programming, debugging, and static analysis.

38. The method of claim 1, comprising filtering email messages and determining any one of junk emails (“

spam”

), illegitimate solicitation emails (“

phishing”

), or legitimate emails in accordance with the policy, wherein the policy is applied in the context of email filtering applications.

39. The method of claim 1, wherein the policy is applied to monitor and control processes in plants or other facilities and to control actions, alerts and other relevant decisions such as those made in a Supervisory Control and Data Acquisition (SCADA) or other system supporting similar functionality.

40. The method of claim 1, wherein the identifying and ranking of potential policy variations based on a score of an objective function includes applying a computational procedure to identify candidate variations and to score the identified candidate policy variations according to the objective function.

41. The method of claim 40, wherein the computational procedure is a computational search procedure.

42. The method of claim 41, wherein the computational search procedure is a meta-heuristic search procedure.

43. The method of claim 41, wherein the computational search procedure is a local search procedure or neighborhood search procedure.

44. The method of claim 42, wherein the computational search procedure is selected from the group consisting of:

a simulated annealing search procedure, a genetic algorithm, a Tabu search procedure, a random optimization procedure, a swarm intelligence algorithm, an evolutionary algorithm.

45. The method of claim 41, wherein the computational procedure is a computational search procedure.

46. The method of claim 40, wherein the computational procedure is selected from the group consisting of:

an optimization algorithm, a non-deterministic computational procedure, a heuristic computational procedure, a computational learning procedure.

47. A computer system for updating a policy that is enforced by a computer program, the computer system comprising:

at least one processor and a memory coupled to the at least one processor, wherein the at least one processor is to;

communicate to at least one user data regarding one or more decisions made by the program over a period of time according to a policy, wherein each decision was made on the particular policy in force at the time the decision was made, and wherein policy data for the policy is stored in a machine readable format in a memory coupled to the computer system;

store in a machine readable format user feedback data indicative of feedback by the at least one user regarding the one or more decisions made by the program according to the policy;

identify and rank one or more potential variations to the policy based on a score of an objective function, wherein the objective function comprises a plurality of function elements, and wherein the plurality of function elements comprises a user satisfaction function element and at least one other function element that measures an extent to which the one or more potential variations are likely to be understandable by the at least one user;

communicate to the at least one user, one or more suggested modifications to the policy based on the ranking of the one or more potential variations to the policy, wherein the one or more suggested modifications can assist the at least one user refine the policy; and

modify the policy data stored in the memory based on one or more selections by the at least one user in response to the communication of the one or more suggested modifications.

48-92. -92. (canceled)

93. An article comprising a computer-readable storage medium comprising executable computer program instructions that when executed enable a computing system to:

communicate to at least one user data regarding one or more decisions made by the program over a period of time according to a policy, wherein each decision was made on the particular policy in force at the time the decision was made, and wherein policy data for the policy is stored in a machine readable format in a memory coupled to the computer system;

store in a machine readable format user feedback data indicative of feedback by the at least one user regarding the one or more decisions made by the program according to the policy;

identify and rank one or more potential variations to the policy based on a score of an objective function, wherein the objective function comprises a plurality of function elements, and wherein the plurality of function elements comprises a user satisfaction function element and at least one other function element that measures an extent to which the one or more potential variations are likely to be understandable by the at least one user;

communicate to the at least one user, one or more suggested modifications to the policy based on the ranking of the one or more potential variations to the policy, wherein the one or more suggested modifications can assist the at least one user refine the policy; and

modify the policy data stored in the memory based on one or more selections by the at least one user in response to the communication of the one or more suggested modifications.