Generating apparatus, generation method, information processing method and program

• US 9,747,616 B2
• Filed: 02/27/2015
• Issued: 08/29/2017
• Est. Priority Date: 03/14/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Osogami, Takayuki
• Primary Examiner(s): Miller, Alan S
• Application Number:
  US14/633,414
• Publication Number:
  US 20150262231A1
• Time in Patent Office: 914 Days
• Field of Search: 705 711- 742
• CPC Class Codes: G06F 17/10 Complex mathematical operations function generation by table look-up G06F1/03; evaluation of elementary functions by calculation G06F7/544
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/0254 based on statistics

Abstract

A generating apparatus generates a set of gain vectors with respect to a transition model having observable visible states and unobservable hidden states and expressing a transition from a present visible state to a subsequent visible state according to an action, the set of gain vectors being generated for each visible state and used for calculation of a cumulative expected gain at and after a reference point in time, the apparatus including a setting section for setting, with respect to each hidden state, a probability distribution over the hidden states for selection used to select vectors to be included in the set of gain vectors from the gain vectors including a component for a cumulative gain, and a selection section for including, in the set of gain vectors, with priority, the gain vector giving the maximum of the cumulative expected gain with respect to the probability distribution for selection.

Claims

1. An apparatus comprising:

a storage device configured to store instructions;

a processing unit communicatively coupled to the storage device and configured to execute the instructions, where the instructions cause the processing unit to;

generate a set of gain vectors with respect to a transition model having observable visible states and unobservable hidden states and expressing a transition from a present visible state to a subsequent visible state according to an action, the set of gain vectors being generated for each visible state and used for calculation of a cumulative expected gain at and after a reference point in time;

wherein the instructions cause the processing unit to generate the set of gain vectors by;

setting, with respect to each hidden state, a probability distribution over the hidden states for selection used to select vectors to be included in the set of gain vectors from the gain vectors including a component for a cumulative gain; and

including, in the set of gain vectors, with priority, the gain vector giving the maximum of the cumulative expected gain with respect to the probability distribution for selection;

wherein the instructions further cause the processing unit to select an optimum action based on the set of gain vectors by;

setting initial conditions for visible and hidden states for an environment to be simulated;

selecting the gain vector which maximizes the cumulative expected gain with respect to the probability distribution over the hidden states at the present point in time;

selecting an action that corresponds to the selected gain vector;

executing the selected action to cause a probabilistic transition from a visible state based on a state transition probability corresponding to the selected action and the present probability distribution over the hidden states; and

updating the probability distribution over the hidden states on the basis of the state transition probability corresponding to the selected action and the present probability distribution over the hidden states.

2. A program product comprising a non-transitory computer readable storage medium having a computer readable program stored thereon, wherein the computer readable program, when executed by a processor, causes the processor to:

generate a set of gain vectors with respect to a transition model having observable visible states and unobservable hidden states and expressing a transition from a present visible state to a subsequent visible state according to an action, the set of gain vectors being generated for each visible state and used for calculation of a cumulative expected gain at and after a reference point in time;

wherein the computer readable program causes the processor to generate the set of gain vectors by

setting, with respect to each hidden state, a probability distribution over the hidden states for selection used to select vectors to be included in the set of gain vectors from the gain vectors including a component for a cumulative gain; and

including, in the set of gain vectors, with priority, the gain vector giving the maximum of the cumulative expected gain with respect to the probability distribution for selection;

wherein the computer readable program further causes the processor to select an optimum action based on the set of gain vectors by;

setting initial conditions for visible and hidden states for an environment to be simulated;

selecting the gain vector which maximizes the cumulative expected gain with respect to the probability distribution over the hidden states at the present point in time;

selecting an action that corresponds to the selected gain vector;

executing the selected action to cause a probabilistic transition from a visible state based on a state transition probability corresponding to the selected action and the present probability distribution over the hidden states; and

updating the probability distribution over the hidden states on the basis of the state transition probability corresponding to the selected action and the present probability distribution over the hidden states.