Controller with artificial intelligence based on selection from episodic memory and corresponding methods

• US 8,660,670 B2
• Filed: 09/07/2010
• Issued: 02/25/2014
• Est. Priority Date: 09/10/2009
• Status: Active Grant

• Current Assignee: Sam Freed
• Original Assignee: Sam Freed
• Inventors: Freed, Sam
• Primary Examiner(s): ORTIZ RODRIGUEZ, CARLOS R
• Application Number:
  US12/876,266
• Publication Number:
  US 20110060425A1
• Time in Patent Office: 1,267 Days
• Field of Search: 700/2, 700/9, 700/20, 700/28, 700/29, 700/30, 700/33, 700/34, 700 47- 51, 700108-110
• US Class Current: 700/49
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 5/043 Distributed expert systems; Blackboards

Abstract

A controller and corresponding method for operating a machine maintains a historical database including at least one sequence of parameters relating to the operating environment, corresponding actions taken, and corresponding outcomes of operation of the machine. The controller searches the database for episodes satisfying relevance criteria relative to a current sequence of parameters and then performs a randomized selection between two or more options, at least one of which is derived from similar episodes from the database. At least one control signal is then output to the machine indicating an action to be performed as determined based on the selected option.

Claims

1. A controller for controlling operation of a machine operating in an operating environment, the machine being responsive to control signals to perform a plurality of actions, the controller receiving signals indicative of parameters relating to the operating environment and sufficient to determine a score relating to an outcome of operation of the machine, the controller comprising:

(a) at least one output for providing control signals to the machine to perform selected actions;

(b) at least one input for receiving signals indicative of parameters relating to the operating environment;

(c) a data storage device containing a historical database including;

at least one sequence of parameters relating to the operating environment;

corresponding actions taken; and

corresponding scores relating to outcomes of operation of the machine; and

(d) a processing system including at least one processor, said processing system being in data communication with said data storage device, said at least one output and said at least one input, said processing system being configured to;

(i) maintaining a list of episodes, corresponding to sequences identified within the historical database, which satisfy a relevance criteria relative to a current sequence of parameters, said relevance criteria employing a relevance function corresponding to an overall measure of similarity between the current sequence of parameters and the episode derived by combining a plurality of similarity functions, values of said relevance function for each episode in said list being updated based on recently input parameters;

(ii) perform a randomized selection between a plurality of actions or sequences of actions, at least one of said actions or sequences of actions being derived from an episode in said list that had a favorable outcome of operation of the machine; and

(iii) outputting at least one control signal to the machine via said at least one output to perform the selected action or sequence of actions.

2. The controller of claim 1, wherein said processing system is configured to perform said randomized selection wherein said randomized selection is weighted towards selection of a similar episode with a most favorable outcome.

3. The controller of claim 1, wherein said processing system is configured to perform said randomized selection between a plurality of actions or sequences of actions including a random action.

4. The controller of claim 1, wherein at least part of said historical database is provided as a pre-stored database containing a simulated history.

5. The controller of claim 1, wherein said processing system is further configured to update said historical database with:

recent sequences of parameters relating to the operating environment;

corresponding actions taken; and

corresponding outcomes of operation of the machine.

6. The controller of claim 1, wherein said processing system is further configured to:

(a) after receipt of new inputs, reevaluate said relevance criteria relative to an updated current sequence of parameters and discard episodes no longer satisfying said relevance criteria; and

(c) intermittently perform a search of said historical database to identify additional episodes satisfying said relevance criteria relative to the updated current sequence of parameters for inclusion in said list.

7. The controller of claim 1, wherein the controlled machine is an additional artificial intelligence controller operating according to a number of operating parameters, and wherein said outputs include a control signal for changing a value of at least one of said operating parameters of said additional artificial intelligence controller.

8. The controller of claim 1, wherein said processing system performs said maintaining a list and said randomized selection according to settings defined by at least one operating parameter, and wherein said processing system is responsive to an input received from an additional artificial intelligence controller to change a value of said at least one operating parameter.

9. The controller of claim 1, further comprising a set of sensors associated with said at least one input and deployed for sensing parameters relating to the operating environment of the machine.

10. The controller of claim 9, wherein said set of sensors includes a range sensor deployed for sensing a distance from at least part of the machine to an object in the operating environment.

11. The controller of claim 9, wherein said set of sensors includes an imaging sensor, and wherein said processing system is further configured to perform image processing on images sampled by said imaging sensor to derive parameters relating to the operating environment.

12. The controller of claim 9, wherein said set of sensors includes an audio input, and wherein said processor system is further configured to perform sound processing on signals sampled by said audio input.

13. The controller of claim 1, wherein the machine is a virtual machine operating in a computer-generated virtual environment.

14. A method for selecting at least one future action to be performed by a machine, the method receiving as an input signals indicative of parameters relating to an operating environment of the machine and sufficient to determine a score relating to an outcome of operation of the machine, the method comprising the steps of

(a) providing a historical database including:

at least one sequence of parameters relating to the operating environment;

corresponding actions taken; and

corresponding scores relating to outcomes of operation of the machine;

(b) maintaining a list of episodes, corresponding to sequences identified within the historical database, which satisfy a relevance criteria relative to a current sequence of parameters, said relevance criteria employing a relevance function corresponding to an overall measure of similarity between the current sequence of parameters and the episode derived by combining a plurality of similarity functions, values of said relevance function for each episode in said list being updated based on recently input parameters;

(c) performing a randomized selection between a plurality of actions or sequences of actions, at least one of said actions or sequences of actions being derived from an episode in said list that had a favorable outcome of operation of the machine; and

(d) outputting to the machine at least one control signal indicative of the selected action or sequence of actions to be performed.

15. The method of claim 14, wherein said randomized selection is weighted towards selection of a similar episode with a most favorable outcome.

16. The method of claim 14, wherein said randomized selection is performed between a plurality of actions or sequences of actions including a random action.

17. The method of claim 14, wherein at least part of said historical database is provided as a pre-stored database containing a simulated history.

18. The method of claim 14, further comprising updating said historical database with:

recent sequences of parameters relating to the operating environment;

corresponding actions taken; and

corresponding outcomes of operation of the machine.

19. The method of claim 14, wherein said maintaining a list of episodes comprises:

(a) after receipt of new inputs, reevaluating said relevance criteria relative to an updated current sequence of parameters and discarding episodes no longer satisfying said relevance criteria; and

(c) intermittently performing a search of said historical database to identify additional episodes satisfying said relevance criteria relative to the updated current sequence of parameters for inclusion in said list.

20. The method of claim 14, wherein the machine is an additional artificial intelligence controller operating according to a number of operating parameters, and wherein said at least one control signal include a control signal for changing a value of at least one of said operating parameters of said additional artificial intelligence controller.

21. The method of claim 14, wherein said maintaining a list and said randomized selection are performed according to settings defined by at least one operating parameter, and wherein a value of said at least one operating parameter is changed in response to an input received from an additional artificial intelligence controller.

22. The method of claim 14, wherein said parameters relating to the operating environment are sensed, at least in part, by a set of sensors deployed for sensing parameters relating to the operating environment of the machine.

23. The method of claim 22, wherein said set of sensors includes a range sensor deployed for sensing a distance from at least part of the machine to an object in the operating environment.

24. The method of claim 22, wherein said set of sensors includes an imaging sensor, and wherein said processing system is further configured to perform image processing on images sampled by said imaging sensor to derive parameters relating to the operating environment.

25. The method of claim 22, wherein said set of sensors includes an audio input, and wherein said processing system is further configured to perform sound processing on signals sampled by said audio input.

26. The method of claim 14, wherein the machine is a virtual machine operating in a computer-generated virtual environment.