Controller with artificial intelligence based on selection from episodic memory and corresponding methods
First Claim
1. A controller for controlling operation of a machine operating in an operating environment, the machine being responsive to control signals to perform a plurality of actions, the controller receiving signals indicative of parameters relating to the operating environment and sufficient to determine a score relating to an outcome of operation of the machine, the controller comprising:
- (a) at least one output for providing control signals to the machine to perform selected actions;
(b) at least one input for receiving signals indicative of parameters relating to the operating environment;
(c) a data storage device containing a historical database including;
at least one sequence of parameters relating to the operating environment;
corresponding actions taken; and
corresponding scores relating to outcomes of operation of the machine; and
(d) a processing system including at least one processor, said processing system being in data communication with said data storage device, said at least one output and said at least one input, said processing system being configured to;
(i) maintaining a list of episodes, corresponding to sequences identified within the historical database, which satisfy a relevance criteria relative to a current sequence of parameters, said relevance criteria employing a relevance function corresponding to an overall measure of similarity between the current sequence of parameters and the episode derived by combining a plurality of similarity functions, values of said relevance function for each episode in said list being updated based on recently input parameters;
(ii) perform a randomized selection between a plurality of actions or sequences of actions, at least one of said actions or sequences of actions being derived from an episode in said list that had a favorable outcome of operation of the machine; and
(iii) outputting at least one control signal to the machine via said at least one output to perform the selected action or sequence of actions.
0 Assignments
0 Petitions
Accused Products
Abstract
A controller and corresponding method for operating a machine maintains a historical database including at least one sequence of parameters relating to the operating environment, corresponding actions taken, and corresponding outcomes of operation of the machine. The controller searches the database for episodes satisfying relevance criteria relative to a current sequence of parameters and then performs a randomized selection between two or more options, at least one of which is derived from similar episodes from the database. At least one control signal is then output to the machine indicating an action to be performed as determined based on the selected option.
-
Citations
26 Claims
-
1. A controller for controlling operation of a machine operating in an operating environment, the machine being responsive to control signals to perform a plurality of actions, the controller receiving signals indicative of parameters relating to the operating environment and sufficient to determine a score relating to an outcome of operation of the machine, the controller comprising:
-
(a) at least one output for providing control signals to the machine to perform selected actions; (b) at least one input for receiving signals indicative of parameters relating to the operating environment; (c) a data storage device containing a historical database including;
at least one sequence of parameters relating to the operating environment;
corresponding actions taken; and
corresponding scores relating to outcomes of operation of the machine; and(d) a processing system including at least one processor, said processing system being in data communication with said data storage device, said at least one output and said at least one input, said processing system being configured to; (i) maintaining a list of episodes, corresponding to sequences identified within the historical database, which satisfy a relevance criteria relative to a current sequence of parameters, said relevance criteria employing a relevance function corresponding to an overall measure of similarity between the current sequence of parameters and the episode derived by combining a plurality of similarity functions, values of said relevance function for each episode in said list being updated based on recently input parameters; (ii) perform a randomized selection between a plurality of actions or sequences of actions, at least one of said actions or sequences of actions being derived from an episode in said list that had a favorable outcome of operation of the machine; and (iii) outputting at least one control signal to the machine via said at least one output to perform the selected action or sequence of actions. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A method for selecting at least one future action to be performed by a machine, the method receiving as an input signals indicative of parameters relating to an operating environment of the machine and sufficient to determine a score relating to an outcome of operation of the machine, the method comprising the steps of
(a) providing a historical database including: - at least one sequence of parameters relating to the operating environment;
corresponding actions taken; and
corresponding scores relating to outcomes of operation of the machine;(b) maintaining a list of episodes, corresponding to sequences identified within the historical database, which satisfy a relevance criteria relative to a current sequence of parameters, said relevance criteria employing a relevance function corresponding to an overall measure of similarity between the current sequence of parameters and the episode derived by combining a plurality of similarity functions, values of said relevance function for each episode in said list being updated based on recently input parameters; (c) performing a randomized selection between a plurality of actions or sequences of actions, at least one of said actions or sequences of actions being derived from an episode in said list that had a favorable outcome of operation of the machine; and (d) outputting to the machine at least one control signal indicative of the selected action or sequence of actions to be performed. - View Dependent Claims (15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26)
- at least one sequence of parameters relating to the operating environment;
Specification