Sparse neural control

• US 10,366,325 B2
• Filed: 12/02/2015
• Issued: 07/30/2019
• Est. Priority Date: 12/07/2011
• Status: Active Grant

• Current Assignee: Artificial Genius Incorporated
• Original Assignee: Paul Burchard
• Inventors: Burchard, Paul
• Primary Examiner(s): Holmes, Michael B
• Application Number:
  US14/956,768
• Publication Number:
  US 20160092764A1
• Time in Patent Office: 1,336 Days
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Aspects herein describe new methods of determining optimal actions to achieve high-level goals with minimum total future cost. At least one high-level goal is inputted into a user device along with various observational data about the world, and a computational unit determines, though a method comprising backward and forward sweeps, an optimal course of action as well as emotions. In one embodiment a user inputs a high-level goal into a cell phone which senses observational data. The cell phone communicates with a server that provides instructions. The server determines an optimal course of action via the method of backward and forward sweeps, and the cell phone then displays the instructions and emotions to the user.

Claims

1. A method comprising:

receiving a plurality of current observations about a real or simulated world, wherein each of the current observations is received from a different one of a plurality of different types of physical sensors;

maintaining, by a computational unit, an objective;

representing the objective using an incremental cost of a plurality of potential actions;

maintaining, by the computational unit, a current uncertainty about an unknown state of a world, wherein the current uncertainty is represented by one or more probabilities of a plurality of high-level explanations of the world, such that a set of possible explanations at any one time is sparse, and wherein the current uncertainty is updated from the plurality of current observations using a filter comprising a sparse network;

determining, by the computational unit, one or more optimal actions to achieve the objective with an optimized expected total future cost, wherein said determining comprises performing both backward induction on the optimized expected total future cost and forward induction on the current uncertainty about the unknown state of the world; and

performing, by a physical actuator, the one or more optimal actions.

2. The method of claim 1, wherein the filter comprises a neural network.

3. The method of claim 1, wherein the filter comprises sparse coding.

4. The method of claim 1, wherein the filter also comprises a stickiness penalty that penalizes changes in resulting possible high-level explanations over time.

5. The method of claim 4, wherein the stickiness penalty linearly penalizes absolute values of the changes, thereby giving preference to locally constant possible explanations with occasional jumps.

6. The method of claim 1, further comprising:

determining a first number t, which represents a future time;

determining a first vector x, which represents a possible explanation at time t;

determining a second vector u, which represents a potential action at time t;

determining a sequence of vectors i, called an archetype, which represents a preferred history of observations from a current time up to time t, consistent with explanation x;

determining a first function ƒ

(t, x), called a local objective function; and

determining a first sparse probability distribution p(t, x), called unconditional uncertainty, which represents an uncertainty of a plurality of explanations x of the world at time t, unconditional on future observations,

wherein said forward and backward induction comprises;

restricting a set of future uncertainties conditional on observations that need to be considered as probability distributions of form p_i(t, x), which comprises a restriction of the unconditional uncertainty p(t, x) to explanations x having the same archetype i at time t;

representing an expected objective value at time t with archetype i as E_pi[ƒ

], where operator E[ ] represents an expectation of a function with respect to a probability distribution;

determining the local objective function ƒ

by backward induction;

determining the unconditional uncertainty p and the archetype i by forward induction; and

determining optimal actions u for each archetype i and time t.

7. The method of claim 6, wherein said forward and backward induction further comprises:

applying the filter, given a current explanation x_t and proposed action u_t, to predict a sparse distribution X(x_t, u_t) of possible explanations at a next time t+1, along with a predicted incremental objective C(x_t, u_t) and a predicted archetype I(x_t, u_t);

determining the backward induction by propagating the local objective function back in time as ƒ

_t(x_t|u_t)=C(x_t, u_t)+E_X(xt,ut)(x)[ƒ

_t+1(x)], where u_t is chosen to optimize an expected objective value

23. The method of claim 1, wherein the performing the one or more optimal actions comprises communicating, by the computational unit and the actuator, via a network.

8. A system comprising:

a physical actuator;

a plurality of physical sensors;

a processor; and

memory storing computer readable instructions that, when executed by the processor, configure the system to perform;

receiving, by the plurality of physical sensors, a current observation about a real or simulated world;

maintaining, by a computational unit, an objective;

representing the objective using an incremental cost of a plurality of potential actions;

maintaining, by the computational unit, a current uncertainty about an unknown state of a world, wherein the current uncertainty is represented by one or more probabilities of a plurality of high-level explanations of the world, such that a set of possible explanations at any one time is sparse, and wherein the current uncertainty is updated from the current observation using a filter comprising a sparse network;

determining, by the computational unit, one or more optimal actions to achieve the objective with an optimized expected total future cost, wherein said determining comprises performing both backward induction on the optimized expected total future cost and forward induction on the current uncertainty about the unknown state of the world; and

causing the physical actuator to perform the one or more optimal actions.

9. The system of claim 8, wherein the filter comprises a neural network.

10. The system of claim 8, wherein the filter comprises sparse coding.

11. The system of claim 8, wherein the filter also comprises a stickiness penalty that penalizes changes in resulting possible high-level explanations over time.

12. The system of claim 11, wherein the stickiness penalty linearly penalizes absolute values of the changes, thereby giving preference to locally constant possible explanations with occasional jumps.

13. The system of claim 8, wherein the memory storing computer readable instructions, when executed by the processor, further configure the system to perform:

determining a first number t, which represents a future time;

determining a first vector x, which represents a possible explanation at time t;

determining a second vector u, which represents a potential action at time t;

determining a sequence of vectors i, called an archetype, which represents a preferred history of observations from a current time up to time t, consistent with explanation x;

determining a first function ƒ

(t, x), called a local objective function; and

determining a first sparse probability distribution p(t, x), called unconditional uncertainty, which represents an uncertainty of a plurality of explanations x of the world at time t, unconditional on future observations,

wherein said forward and backward induction comprises;

restricting a set of future uncertainties conditional on observations that need to be considered as probability distributions of form p_i(t, x), which comprises a restriction of the unconditional uncertainty p(t, x) to explanations x having the same archetype i at time t;

representing an expected objective value at time t with archetype i as E_pi[ƒ

], where operator E[ ] represents an expectation of a function with respect to a probability distribution;

determining the local objective function ƒ

by backward induction;

determining the unconditional uncertainty p and the archetype i by forward induction; and

determining optimal actions u for each archetype i and time t.

14. The system of claim 13, wherein said forward and backward induction further comprises:

applying the filter, given a current explanation x_t and proposed action u_t, to predict a sparse distribution X(x_t, u_t) of possible explanations at a next time t+1, along with a predicted incremental objective C(x_t, u_t) and a predicted archetype I(x_t, u_t);

determining the backward induction by propagating the local objective function back in time as ƒ

_t(x_t|u_t)=C(x_t, u_t)+E_X(xt,ut)(x)[ƒ

_t+1(x)], where u_t is chosen to optimize an expected objective value

24. The system of claim 8, wherein the physical actuator comprises an automated personal assistant.

15. One or more non-transitory computer readable media comprising computer readable instructions stored thereon, wherein when the instructions are executed by a data processing system, said instructions configure the data processing system to perform:

receiving, by a plurality of physical sensors, a plurality of observations about a real or simulated world;

maintaining, by a computational unit, an objective;

representing the objective using an incremental cost of a plurality of potential actions;

maintaining, by the computational unit, a current uncertainty about an unknown state of a world, wherein the current uncertainty is represented by one or more probabilities of a plurality of high-level explanations of the world, such that a set of possible explanations at any one time is sparse, and wherein the current uncertainty is updated from the plurality of observations using a filter comprising a sparse network;

determining, by the computational unit, one or more optimal actions to achieve the objective with an optimized expected total future cost, wherein said determining comprises performing both backward induction on the optimized expected total future cost and forward induction on the current uncertainty about the unknown state of the world;

causing a physical actuator to output instructions for performing the one or more optimal actions.

16. The one or more non-transitory computer readable media of claim 15, wherein the filter comprises a neural network.

17. The one or more non-transitory computer readable media of claim 15, wherein the filter comprises sparse coding.

18. The one or more non-transitory computer readable media of claim 15, wherein the filter comprises one or more Boltzmann machines.

19. The one or more non-transitory computer readable media of claim 15, wherein the filter also comprises a stickiness penalty that penalizes changes in resulting possible high-level explanations over time.

20. The one or more non-transitory computer readable media of claim 19, wherein the stickiness penalty linearly penalizes absolute values of the changes, thereby giving preference to locally constant possible explanations with occasional jumps.

21. The one or more non-transitory computer readable media of claim 15, wherein when the instructions are executed by a data processing system, said instructions configure the data processing system to further perform:

determining a first number t, which represents a future time;

determining a first vector x, which represents a possible explanation at time t;

determining a second vector u, which represents a potential action at time t;

determining a sequence of vectors i, called an archetype, which represents a preferred history of observations from a current time up to time t, consistent with explanation x;

determining a first function ƒ

(t, x), called a local objective function; and

determining a first sparse probability distribution p(t, x), called unconditional uncertainty, which represents an uncertainty of a plurality of explanations x of the world at time t, unconditional on future observations,

wherein said forward and backward induction comprises;

restricting a set of future uncertainties conditional on observations that need to be considered as probability distributions of form p_i(t, x), which comprises a restriction of the unconditional uncertainty p(t, x) to explanations x having the same archetype i at time t;

representing an expected objective value at time t with archetype i as E_pi[ƒ

], where operator E[ ] represents an expectation of a function with respect to a probability distribution;

determining the local objective function ƒ

by backward induction;

determining the unconditional uncertainty p and the archetype i by forward induction; and

determining optimal actions u for each archetype i and time t.

22. The one or more non-transitory computer readable media of claim 21, wherein said forward and backward induction further comprises:

applying the filter, given a current explanation x_t and proposed action u_t, to predict a sparse distribution X(x_t, u_t) of possible explanations at a next time t+1, along with a predicted incremental objective C(x_t, u_t) and a predicted archetype I(x_t, u_t);

determining the backward induction by propagating the local objective function back in time as ƒ

_t(x_t|u_t)=C(x_t, u_t)+E_X(xt,ut)(x)[ƒ

_t+1(x)], where u_t is chosen to optimize an expected objective value