Predictive action decision device and action decision method
First Claim
Patent Images
1. A predictive action determination apparatus comprising:
- a state observation section for observing a state with respect to a predetermined environment and obtaining state data;
a state value storage section for storing a state value for each of states of the environment;
an environment prediction section for predicting a future state change in the environment, based on the state data obtained by the state observation section;
a target state determination section for determining, as a target state, a future state suitable for action determination among future states predicted by the environment prediction section, based on the state value for each of future states stored in the state value storage section; and
a first action determination section for determining an action of the apparatus, based on the target state determined by the target state determination section, wherein the environment prediction section predicts a future state change in the environment, which is not influenced by actions of the apparatus.
1 Assignment
0 Petitions
Accused Products
Abstract
In a predictive action determination apparatus (10), a state observation section (12) observes a state with respect to an environment (11) and obtains state data s(t). An environment prediction section (13) predicts, based on the state data s(t), a future state change in the environment. A target state determination section (15) determines, as a target state, a future state suitable for action determination with reference to a state value storage section (14). A prediction-based action determination section (16) determines an action based on a determined target state.
-
Citations
15 Claims
-
1. A predictive action determination apparatus comprising:
-
a state observation section for observing a state with respect to a predetermined environment and obtaining state data;
a state value storage section for storing a state value for each of states of the environment;
an environment prediction section for predicting a future state change in the environment, based on the state data obtained by the state observation section;
a target state determination section for determining, as a target state, a future state suitable for action determination among future states predicted by the environment prediction section, based on the state value for each of future states stored in the state value storage section; and
a first action determination section for determining an action of the apparatus, based on the target state determined by the target state determination section, wherein the environment prediction section predicts a future state change in the environment, which is not influenced by actions of the apparatus. - View Dependent Claims (3, 4, 6, 7, 8, 9, 10)
-
-
2. (canceled)
-
5. A predictive action determination apparatus comprising:
-
a state observation section for observing a state with respect to a predetermined environment and obtaining state data;
a state value storage section for storing a state value for each of states of the environment;
an environment prediction section for predicting a future state change in the environment, based on the state data obtained by the state observation section;
a target state determination section for determining, as a target state, a future state suitable for action determination among future states predicted by the environment prediction section, based on the state value for each of future states stored in the state value storage section; and
a first action determination section for determining an action of the apparatus, based on the target state determined by the target state determination section, wherein the target state determination section discounts the state value obtained from the state value storage section according to the number of steps from a current step and uses the discounted state value.
-
-
11. A predictive action determination apparatus comprising:
-
a state observation section for observing a state with respect to a predetermined environment and obtaining state data;
a state value storage section for storing a state value for each of states of the environment;
an environment prediction section for predicting a future state change in the environment, based on the state data obtained by the state observation section;
a target state determination section for determining, as a target state, a future state suitable for action determination among future states predicted by the environment prediction section, based on the state value for each of future states stored in the state value storage section; and
a first action determination section for determining an action of the apparatus, based on the target state determined by the target state determination section, wherein the environment prediction section includes;
a state change detection section for receiving the state data and detecting a state in a previous step from a current state indicated by the state data;
a state change storage section for storing, as a state change, a combination of the current state and the state in the previous step detected by the state change detection section; and
a state prediction section for predicting a state after the current state from the state change storage section.
-
-
12. A method of determining in a predictive action determination apparatus an action of the apparatus, comprising:
-
a first step of observing a state with respect to a predetermined environment and obtaining state data;
a second step of predicting a future state change in the environment, based on the obtained state data;
a third step of determining, as a target state, a future state suitable for action determination among predicted future states, with reference to the state value for each of the future states; and
a fourth step of determining the action of the apparatus, based on the determined target state, wherein a predicted state change is a future state change in the environment, which is not influenced by actions of the apparatus. - View Dependent Claims (14, 15)
-
-
13. (canceled)
Specification