CONTROLLER WITH ARTIFICIAL INTELLIGENCE BASED ON SELECTION FROM EPISODIC MEMORY AND CORRESPONDING METHODS

• US 20110060425A1
• Filed: 09/07/2010
• Published: 03/10/2011
• Est. Priority Date: 09/10/2009
• Status: Active Grant

• Current Assignee: Sam Freed
• Original Assignee: Sam Freed
• Inventors: FREED, Sam
• Granted Patent: US 8,660,670 B2
• Time in Patent Office: Days
• US Class Current: 700/34
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 5/043 Distributed expert systems; Blackboards

Abstract

A controller and corresponding method for operating a machine maintains a historical database including at least one sequence of parameters relating to the operating environment, corresponding actions taken, and corresponding outcomes of operation of the machine. The controller searches the database for episodes satisfying relevance criteria relative to a current sequence of parameters and then performs a randomized selection between two or more options, at least one of which is derived from similar episodes from the database. At least one control signal is then output to the machine indicating an action to be performed as determined based on the selected option.

Claims

1. A controller for controlling operation of a machine operating in an operating environment, the machine being responsive to control signals to perform a plurality of actions, the controller receiving signals indicative of parameters relating to the operating environment and sufficient to determine an outcome of operation of the machine, the controller comprising:

(a) at least one output for providing control signals to the machine to perform selected actions;

(b) at least one input for receiving signals indicative of parameters relating to the operating environment;

(c) a data storage device containing a historical database including at least one sequence of parameters relating to the operating environment, corresponding actions taken, and corresponding outcomes of operation of the machine; and

(d) a processing system including at least one processor, said processing system being in data communication with said data storage device, said at least one output and said at least one input, said processing system being configured to;

(i) search within said historical database for similar episodes within said at least one sequence of parameters which satisfy relevance criteria relative to a current sequence of parameters;

(ii) perform a randomized selection between a plurality of options, at least one of said options being derived from one of said similar episodes that had a favorable outcome of operation of the machine; and

(iii) outputting at least one control signal to the machine via said output to perform a selected action, said control signal being determined as a function of a selected one of said options.

2. The controller of claim 1, wherein said processing system is configured to perform said randomized selection wherein said randomized selection is weighted towards selection of a similar episode with a most favorable outcome.

3. The controller of claim 1, wherein said processing system is configured to perform said randomized selection between a plurality of options including a random action option.

4. The controller of claim 1, wherein at least part of said historical database is provided as a pre-stored database containing a simulated history.

5. The controller of claim 1, wherein said processing system is further configured to update said historical database with recent sequences of parameters relating to the operating environment, corresponding actions taken, and corresponding outcomes of operation of the machine.

6. The controller of claim 1, wherein said processing system is further configured to:

(a) maintain a list of episodes which satisfy said relevance criteria relative to a current sequence of parameters;

(b) after receipt of new inputs, reevaluate said relevance criteria relative to the updated current sequence of parameters and discard episodes no longer satisfying said relevance criteria; and

(c) intermittently perform a search of said historical database to identify additional episodes of sequences of parameters satisfying said relevance criteria relative to the updated current sequence of parameters for inclusion in said list,

and wherein said randomized selection is performed using options derived from episodes in said list.

7. The controller of claim 1, wherein the controlled machine is an additional artificial intelligence controller operating according to a number of operating parameters, and wherein said outputs include a control signal for changing a value of at least one of said operating parameters of said additional artificial intelligence controller.

8. The controller of claim 1, wherein said processing system performs said searching and said randomized selection according to settings defined by at least one operating parameter, and wherein said processing system is responsive to an input received from an additional artificial intelligence controller to change a value of said at least one operating parameter.

9. The controller of claim 1, further comprising a set of sensors associated with said at least one input and deployed for sensing parameters relating to the operating environment of the machine.

10. The controller of claim 9, wherein said set of sensors includes a range sensor deployed for sensing a distance from at least part of the machine to an object in the operating environment.

11. The controller of claim 9, wherein said set of sensors includes an imaging sensor, and wherein said processor system is further configured to perform image processing on imaged sampled by said imaging sensor to derive parameters relating to the operating environment.

12. The controller of claim 9, wherein said set of sensors includes an audio input, and wherein said processor system is further configured to perform sound processing on signals sampled by said audio input.

13. The controller of claim 1, wherein the machine is a virtual machine operating in a computer-generated virtual environment.

14. A method for selecting at least one future action to be performed by a machine, the method comprising the steps of

(a) providing a historical database including at least one sequence of parameters relating to the operating environment, corresponding actions taken, and corresponding outcomes of operation of the machine;

(b) searching within said historical database for similar episodes within said at least one sequence of parameters which satisfy relevance criteria relative to a current sequence of parameters;

(c) performing a randomized selection between a plurality of options, at least one of said options being derived from one of said similar episodes that had a favorable outcome of operation of the machine; and

(d) outputting to the machine at least one control signal indicative of at least one action to be performed, said at least one action being determined as a function of a selected one of said options.

15. The method of claim 14, wherein said randomized selection is weighted towards selection of a similar episode with a most favorable outcome.

16. The method of claim 14, wherein said randomized selection is performed between a plurality of options including a random action option.

17. The method of claim 14, wherein at least part of said historical database is provided as a pre-stored database containing a simulated history.

18. The method of claim 14, further comprising updating said historical database with recent sequences of parameters relating to the operating environment, corresponding actions taken, and corresponding outcomes of operation of the machine.

19. The method of claim 14, further comprising:

(a) maintaining a list of episodes which satisfy said relevance criteria relative to a current sequence of parameters;

(b) after receipt of new inputs, reevaluating said relevance criteria relative to the updated current sequence of parameters and discarding episodes no longer satisfying said relevance criteria; and

(c) intermittently performing a search of said historical database to identify additional episodes of sequences of parameters satisfying said relevance criteria relative to the updated current sequence of parameters for inclusion in said list,

and wherein said randomized selection is performed using options derived from episodes in said list.

20. The method of claim 14, wherein the controlled machine is an additional artificial intelligence controller operating according to a number of operating parameters, and wherein said outputs include a control signal for changing a value of at least one of said operating parameters of said additional artificial intelligence controller.

21. The method of claim 14, wherein said searching and said randomized selection are performed according to settings defined by at least one operating parameter, and wherein a value of said at least one operating parameter is changed in response to an input received from an additional artificial intelligence controller.

22. The method of claim 14, wherein said parameters relating to the operating environment are sensed, at least in part, by a set of sensors deployed for sensing parameters relating to the operating environment of the machine.

23. The method of claim 22, wherein said set of sensors includes a range sensor deployed for sensing a distance from at least part of the machine to an object in the operating environment.

24. The method of claim 22, wherein said set of sensors includes an imaging sensor, and wherein said processor system is further configured to perform image processing on imaged sampled by said imaging sensor to derive parameters relating to the operating environment.

25. The method of claim 22, wherein said set of sensors includes an audio input, and wherein said processor system is further configured to perform sound processing on signals sampled by said audio input.

26. The method of claim 14, wherein the machine is a virtual machine operating in a computer-generated virtual environment.

27. A controller for controlling operation of a machine operating in an operating environment, the machine being responsive to control signals to perform a plurality of actions, the controller receiving signals indicative of parameters relating to the operating environment and sufficient to determine an outcome of operation of the machine, the controller comprising:

(a) a data storage device;

(b) at least one output for providing control signals to the machine to perform selected actions;

(c) at least one input for receiving signals indicative of parameters relating to the operating environment; and

(d) a processing system including at least one processor, said processing system being in data communication with said data storage device, said at least one output and said at least one input, said processing system being configured to;

(i) process a current sequence of parameters together with data from said data storage device to determine a plurality of options satisfying a desirability criterion;

(ii) perform a randomized selection between said plurality of options; and

(iii) outputting at least one control signal to the machine via said output to perform a selected action, said control signal being determined as a function of a selected one of said options.

28. A method for selecting at least one future action to be performed by a machine, the method comprising the steps of

(a) processing a current sequence of parameters together with data from said data storage device to determine a plurality of options satisfying a desirability criterion;

(b) performing a randomized selection between said plurality of options; and

(c) outputting to the machine at least one control signal indicative of at least one action to be performed, said at least one action being determined as a function of a selected one of said options.