Method for performing a plurality of candidate actions and monitoring the responses so as to choose the next candidate action to take to control a system so as to optimally control its objective function
First Claim
1. A method of controlling a system to optimize an objective function thereof, the system performing a plurality of candidate actions and monitoring response performances of a performance of a respective candidate action, where the objective function is a function of the monitored response performances following decisions and actions taken, the method comprising the steps of:
- a) monitoring response performance of a respective candidate action that is chosen to be performed by the system;
b) storing, according to the candidate action performed by the system, a representation of said monitored response performance;
c) calculating the expected growth in regret associated with each of the plurality of candidate actions, assessed using a probability distribution based on the historical response performances to date of said plurality of candidate actions, where the expected growth in regret is a system performance measure that is calculated to represent the trade-off between the relative merit of exploration of one or more apparently non-best candidate actions to mitigate the risk of ignoring one of said one or more apparently non-best candidate actions which may actually be the current best candidate action, with respect to the relative merit of exploiting what appears to be the current best candidate action but which in fact may not be the current best candidate action, based on said historical response performances to date;
d) choosing as the next action the candidate action that is calculated to result in the lowest expected growth in regret after the chosen candidate action is performed by the system;
e) commanding the system to perform the chosen next action; and
f) repeating steps a) to e) to control the system so as to substantially optimize the objective function of the system.
6 Assignments
0 Petitions
Accused Products
Abstract
The present disclosure relates to a controller for controlling a system, capable of presentation of a plurality of candidate propositions resulting in a response performance, in order to optimise an objective function of the system. The controller has a means for storing, according to candidate proposition, a representation of the response performance in actual use of respective propositions; means for assessing which candidate proposition is likely to result in the lowest expected regret after the next presentation on the basis of an understanding of the probability distribution of the response performance of all of the plurality of candidate propositions; where regret is a term used for the shortfall in response performance between always presenting a true best candidate proposition and using the candidate proposition actually presented.
65 Citations
15 Claims
-
1. A method of controlling a system to optimize an objective function thereof, the system performing a plurality of candidate actions and monitoring response performances of a performance of a respective candidate action, where the objective function is a function of the monitored response performances following decisions and actions taken, the method comprising the steps of:
-
a) monitoring response performance of a respective candidate action that is chosen to be performed by the system; b) storing, according to the candidate action performed by the system, a representation of said monitored response performance; c) calculating the expected growth in regret associated with each of the plurality of candidate actions, assessed using a probability distribution based on the historical response performances to date of said plurality of candidate actions, where the expected growth in regret is a system performance measure that is calculated to represent the trade-off between the relative merit of exploration of one or more apparently non-best candidate actions to mitigate the risk of ignoring one of said one or more apparently non-best candidate actions which may actually be the current best candidate action, with respect to the relative merit of exploiting what appears to be the current best candidate action but which in fact may not be the current best candidate action, based on said historical response performances to date; d) choosing as the next action the candidate action that is calculated to result in the lowest expected growth in regret after the chosen candidate action is performed by the system; e) commanding the system to perform the chosen next action; and f) repeating steps a) to e) to control the system so as to substantially optimize the objective function of the system. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A system having means for performing a plurality of candidate actions and means for monitoring response performances of a performance of a respective candidate action during performance of an objective function of the system, where the objective function is a function of the monitored response performances following decisions and actions taken, the system further having a control apparatus that is programmed to control the objective function of the system by performing the method comprising the steps of:
-
a) monitoring response performance of a respective candidate action that is chosen to be performed by the system; b) storing, according to the candidate action performed by the system, a representation of said monitored response performance; c) calculating the expected growth in regret associated with each of the plurality of candidate actions, assessed using a probability distribution based on the historical response performances to date of said plurality of candidate actions, where the expected growth in regret is a system performance measure that is calculated to represent the trade-off between the relative merit of exploration of one or more apparently non-best candidate actions to mitigate the risk of ignoring one of said one or more apparently non-best candidate actions which may actually be the current best candidate action, with respect to the relative merit of exploiting what appears to be the current best candidate action but which in fact may not be the current best candidate action, based on said historical response performances to date; d) choosing as the next action the candidate action that is calculated to result in the lowest expected growth in regret after the chosen candidate action is performed; e) commanding the system to perform the chosen next action; and f) repeating steps a) to e) to control the system so as to substantially optimize the objective function of the system. - View Dependent Claims (13)
-
-
14. A control apparatus for controlling a system to optimize an objective function thereof, the system performing a plurality of candidate actions and monitoring response performances of a performance of a respective candidate action, where the objective function is a function of the monitored response performances following decisions and actions taken, the control apparatus comprising
a) means for monitoring response performance of a respective candidate action that is chosen to be performed by the system; -
b) means for storing, according to the candidate action performed by the system, a representation of said monitored response performance; c) means for calculating the expected growth in regret associated with each of the plurality of candidate actions, assessed using a probability distribution based on the historical response performances to date of said plurality of candidate actions, where the expected growth in regret is a system performance measure that is calculated to represent the trade-off between the relative merit of exploration of one or more apparently non-best candidate actions to mitigate the risk of ignoring one of said one or more apparently non-best candidate actions which may actually be the current best candidate action, with respect to the relative merit of exploiting what appears to be the current best candidate action but which in fact may not be the current best candidate action, based on said historical response performances to date; d) means for choosing as the next action the candidate actions that is calculated to result in the lowest expected growth in regret after the chosen candidate action is performed by the system; and e) means for commanding the system to perform the chosen next action, wherein the control apparatus controls the system so as to substantially optimize the objective function of the system.
-
-
15. A method of controlling a system with two or more subsystems to optimize an objective function of the system, the system performing a plurality of candidate actions, wherein a candidate action is represented by the selection of a lower level subsystem from said two or more subsystems, and wherein the system monitors the response performance of the selected subsystem, where the objective function is a function of the monitored response performances following decisions and actions taken, the method comprising the steps of:
-
a) monitoring response performance of a respective candidate action that is chosen to be performed by the system; b) storing, according to the candidate action performed by the system, a representation of said monitored subsystem performance in response to the candidate action; c) calculating the expected growth in regret associated with each of the plurality of candidate actions, assessed using a probability distribution based on the historical response performances to date of said plurality of candidate actions, where the expected growth in regret is a system performance measure that is calculated to represent the trade-off between the relative merit of exploration of one or more apparently non-best candidate actions to mitigate the risk of ignoring one of said one or more apparently non-best candidate actions which may actually be the current best candidate action, with respect to the relative merit of exploiting what appears to be the current best candidate action but which in fact may not be the current best candidate action, based on said historical response performances to date; d) choosing as the next action the candidate action that is calculated to result in the lowest expected growth in regret after the chosen candidate action is performed by the system; e) commanding the system to perform the chosen next action using a corresponding lower level subsystem; and f) repeating steps a) to e) to control the system so as to substantially optimize the objective function of the system.
-
Specification