Risk Quantification for Policy Deployment
First Claim
1. In a digital medium environment for identifying and deploying potential digital advertising campaigns, where campaigns can be altered, removed, or replaced on demand, a method for optimizing campaign selection in the digital medium environment, the method comprising:
- receiving a policy at one or more computing devices, the policy configured for deployment by a content provider to select advertisements; and
controlling deployment, by the one or more computing devices, of the received policy by the content provider based at least in part on a quantification of risk that is likely involved in the deployment of the received policy as opposed to a deployed policy of the content provider, the controlling including;
applying reinforcement learning and a concentration inequality on deployment data that describes the deployment of the deployed policy by the content provider to estimate values of a measure of performance of the received policy and to quantify the risk by calculating one or more statistical guarantees of the estimated values; and
causing deployment of the received policy responsive to a determination that the one or more statistical guarantees express at least a confidence level that the estimated values of the measure of performance at least correspond to a threshold that is based at least in part on a measure of performance of the deployed policy by the content provider.
2 Assignments
0 Petitions
Accused Products
Abstract
Risk quantification, policy search, and automated safe policy deployment techniques are described. In one or more implementations, techniques are utilized to determine safety of a policy, such as to express a level of confidence that a new policy will exhibit an increased measure of performance (e.g., interactions or conversions) over a currently deployed policy. In order to make this determination, reinforcement learning and concentration inequalities are utilized, which generate and bound confidence values regarding the measurement of performance of the policy and thus provide a statistical guarantee of this performance. These techniques are usable to quantify risk in deployment of a policy, select a policy for deployment based on estimated performance and a confidence level in this estimate (e.g., which may include use of a policy space to reduce an amount of data processed), used to create a new policy through iteration in which parameters of a policy are iteratively adjusted and an effect of those adjustments are evaluated, and so forth.
-
Citations
20 Claims
-
1. In a digital medium environment for identifying and deploying potential digital advertising campaigns, where campaigns can be altered, removed, or replaced on demand, a method for optimizing campaign selection in the digital medium environment, the method comprising:
-
receiving a policy at one or more computing devices, the policy configured for deployment by a content provider to select advertisements; and controlling deployment, by the one or more computing devices, of the received policy by the content provider based at least in part on a quantification of risk that is likely involved in the deployment of the received policy as opposed to a deployed policy of the content provider, the controlling including; applying reinforcement learning and a concentration inequality on deployment data that describes the deployment of the deployed policy by the content provider to estimate values of a measure of performance of the received policy and to quantify the risk by calculating one or more statistical guarantees of the estimated values; and causing deployment of the received policy responsive to a determination that the one or more statistical guarantees express at least a confidence level that the estimated values of the measure of performance at least correspond to a threshold that is based at least in part on a measure of performance of the deployed policy by the content provider. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A system comprising:
one or more computing devices configured to perform operations including controlling deployment of a received policy based at least in part on a quantification of risk that is likely involved in the deployment of the received policy as opposed to a deployed policy, the controlling including; using reinforcement learning and a concentration inequality on deployment data that describes the deployment of the deployed policy to estimate values of a measure of performance of the received policy and to quantify the risk by calculating one or more statistical guarantees regarding the estimated values; and causing replacement of the deployment of the deployed policy with the received policy responsive to a determination that the one or more statistical guarantees express at least a confidence level that the estimated values of the measure of performance at least correspond to a threshold that is based at least in part on a measure of performance of the deployed policy. - View Dependent Claims (14, 15, 16)
-
17. A content provider comprising one or more computing devices configured to perform operations including:
-
deploying a policy to select advertisements to be included with content based on one or more characteristics associated with a request for the content; and replacing the deployed policy with another policy that is selected through use of reinforcement learning and a concentration inequality to process deployment data and determine that the one or more statistical guarantees express at least a confidence level that estimated values of a measure of performance of the received policy at least corresponds to a threshold that is based at least in part on a measure of performance of the deployed policy. - View Dependent Claims (18, 19, 20)
-
Specification