Information processing apparatus, information processing method, and computer program for controlling state transition

• US RE46,186 E1
• Filed: 06/26/2013
• Issued: 10/25/2016
• Est. Priority Date: 03/13/2008
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sabe, Kohtaro, Minamino, Katsuki, Kawamoto, Kenta, Suzuki, Hirotaka, Hidai, Kenichi
• Primary Examiner(s): Basehoar, Adam L
• Application Number:
  US13/927,708
• Time in Patent Office: 1,217 Days
• Field of Search: 706/14, 706/25, 706/19, 706/22, 706/55, 706/58, 706/59, 700/94, 715/716, 7042561-2568
• US Class Current: 1/1
• CPC Class Codes: G05B 13/0265 the criterion being a learning criterion
• CPC Class Codes: G05B 19/045 using logic state machines, consisting only of a memory or a programmable logic device containing the logic for the controlled machine and in which the state of its outputs is dependent on the state of its inputs or part of its own output states, e.g. binary decision controllers, finite state controllers
• CPC Class Codes: G05B 2219/23288 Adaptive states; learning transitions
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 99/005 Learning machines, i.e. computer in which a programme is changed according to experience gained by the machine itself during a complete run neural networks G06N3/02; knowledge based models G06N5/00; fuzzy logic systems G06N7/02; adaptive control systems G05B13/00
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]

Abstract

An information processing apparatus includes: model learning means for self-organizing, on the basis of a state transition model having a state and state transition to be learned by using time series data as data in time series, an internal state from an observation signal obtained by a sensor; and controller learning means for performing learning for allocating a controller, which outputs an action, to each of transitions of a state or each of transition destination states in the state transition model indicating the internal state self-organized by the model learning means.

Claims

1. An information processing apparatus comprising:

model learning means for self-organizing, on the basis of a state transition model having a state and state transition to be learned by using time series data as data in time series, an internal state from an observation signal obtained by a sensor;

controller learning means for performing learning for allocating a controller, which outputs an action, to each of transitions of a state or each of transition destination states in the state transition model indicating the internal state self-organized by the model learning means;

initial-structure setting means for initializing structure of the state transition model to sparse structure;

data adjusting means for adjusting the time series data used for the learning according to progress of the learning and outputting time series data after the adjustment;

parameter estimating means for estimating a parameter of the state transition model using the time series data after adjustment; and

structure adjusting means for adjusting the structure of the state transition model.

2. An information processing apparatus according to claim 1, further comprising:

planning means for planning a path for attaining a target as a transition sequence of a state on the state transition model indicating the internal state self-organized by the model learning means; and

execution managing means for invoking, for each of transitions included in the path planned by the planning means, the controller allocated by the controller learning means to manage execution of an action along the path.

3. An information processing apparatus according to claim 2, wherein

the model learning means self-organizes, independently for each of plural modals, an internal state from an observation signal obtained by a sensor of a modal corresponding thereto on the basis of state transition models, and

the information processing apparatus further includes causality means for estimating causality of transition in one state transition model and a state of another state transition model among the state transition models for each of the plural modals respectively indicating the internal state self-organized by the model learning means.

4. An information processing apparatus according to claim 3, wherein the execution managing means causes, when it is difficult to directly control an internal state of a predetermined modal among the plural modals respectively indicating the internal state self-organized by the model learning means, the planning means to recursively execute planning to control the internal state on the basis of the causality estimated by the causality means.

5. An information processing apparatus according to claim 2, further comprising setting means for spontaneously setting a target from the internal state self-organized by the model learning means, wherein

the controller learning means, the planning means, and the execution managing means execute respective kinds of processing to realize the target spontaneously set by the setting means.

7. An information processing apparatus according to claim 1, wherein the data adjusting means adjusts, according to the progress of the learning, the time series data from data including a macro characteristic to data including a micro characteristic.

8. An information processing apparatus according to claim 1, wherein the structure adjusting means adjusts the structure of the state transition model by performing division of a state of the state transition model, merging of a state, the addition of a state, addition of state transition, deletion of a state, or deletion of state transition.

9. An information processing apparatus according to claim 1, further comprising evaluating means for evaluating the state transition model for which the learning is performed and determining, on the basis of a result of the evaluation of the state transition model, whether the learning should be finished.

10. An information processing apparatus according to claim 1, wherein the state transition model is an HMM (Hidden Markov Model).

11. An information processing apparatus according to claim 10, wherein the structure adjusting means performs deletion of a state with a state not forming a path calculated by a Viterbi method among states of the HMM set as a target.

12. An information processing apparatus according to claim 10, wherein the structure adjusting means performs deletion of state transition with state transition not forming a path calculated by a Viterbi method among state transitions of the HMM set as a target.

13. An information processing apparatus according to claim 1, further comprising:

detecting means for detecting an event that occurs immediately preceding state transition that occurs in a first set including events exclusive to one another, the event being an event in a second set, which is a single or plural other sets including events exclusive to one another; and

estimating means estimating, with the state transition set as a result event and the event in the second set detected by the detecting means set as a cause event, causality between the events included in the different sets.

14. An information processing apparatus according to claim 13, wherein the estimating means calculates, for each of events that occur in the second set immediately preceding the state transition and are detected by the detecting means, a conditional probability concerning the state transition and estimates causality between the events included in the different sets.

15. An information processing apparatus according to claim 14, wherein

the detecting means detects, for a first event immediately preceding occurrence of the state transition and a second event in the second set that occurs simultaneously with the first event, a first number of times the first and second events simultaneously occur immediately preceding the state transition and a second number of times the first and second events simultaneously occurs, and

the estimating means calculates a conditional probability concerning the state transition by dividing the first number of times detected by the detecting means by the second number of times detected by the detecting means.

16. An information processing apparatus according to claim 14, further comprising storing means for storing, in association with each other, each of the events that occur in the second set immediately preceding the state transition and the conditional probability concerning the state transition calculated for each of the events by the estimating means.

17. An information processing apparatus according to claim 16, further comprising determining means for determining, as a behavior for causing the state transition, a behavior for realizing an event that occurs in the second set immediately preceding the state transition, the event being associated with the conditional probability that is highest or equal to or higher than a fixed probability.

18. An information processing apparatus according to claim 16, further comprising controlling means for controlling, on the basis of a number of times of simultaneous occurrence of a first even immediately preceding occurrence of the state transition and a second event in the second set that occur simultaneously with the first event, granularity of the events stored by the storing means in association with the conditional probability concerning the state transition.

6. An information processing method comprising the steps of:

self-organizing, on the basis of a state transition model having a state and state transition to be learned by using time series data as data in time series, an internal state from an observation signal obtained by a sensor;

performing learning for allocating a controller, which outputs an action, to each of transitions of a state or each of transition destination states in the state transition model indicating the self-organized internal state self-organized;

initializing structure of the state transition model to sparse structure;

adjusting the time series data used for the learning according to progress of the learning and outputting time series data after the adjustment;

estimating a parameter of the state transition model using the time series data after adjustment; and

adjusting the structure of the state transition model.

19. An information processing apparatus comprising:

a model learning unit self-organizing, on the basis of a state transition model having a state and state transition to be learned by using time series data as data in time series, an internal state from an observation signal obtained by a sensor;

a controller learning unit performing learning for allocating a controller, which outputs an action, to each of transitions of a state or each of transition destination states in the state transition model indicating the internal state self-organized by the model learning unit;

an initial-structure setting unit initializing structure of the state transition model to sparse structure;

a data adjusting unit adjusting the time series data used for the learning according to progress of the learning and outputting time series data after the adjustment;

a parameter estimating unit estimating a parameter of the state transition model using the time series data after adjustment; and

a structure adjusting unit adjusting the structure of the state transition model.

20. An information processing apparatus comprising:

a data receiving unit that receives time series data for learning;

a model storing unit that stores a state transition model for modeling the time series data, the state transition model including state information and state transition information;

an initial-structure setting unit that initializes structure of the state transition model to sparse structure;

a parameter estimating unit that estimates a parameter of the state transition model using the time series data; and

a structure adjusting unit that adjusts the structure of the state transition model by, with reference to a first state included in the structure of the state transition model and set as a target, removing a second merging state, in which the second merging state is included in the structure of the state transition model, from the structure of the state transition model.

21. The information processing apparatus of claim 20, wherein the state transition model is based on an HMM (Hidden Markov Model).

22. The information processing apparatus of claim 20, further comprising:

a data adjusting unit that adjusts the time series data according to progress of learning.

23. The information processing apparatus of claim 22, wherein the structure adjusting unit adjusts the structure based on adjustment conducted by the data adjusting unit.

24. The information processing apparatus of claim 22, wherein the data adjusting unit adjusts the time series data by adjusting a sampling rate for the time series data.

25. The information processing apparatus of claim 20, wherein the structure adjusting unit adjusts the structure by at least limiting a transition of the state transition model.

26. The information processing apparatus of claim 20, wherein the structure of the state transition model is adjusted by the removing of the second merging state and adding, with reference to the first state, a third state to the structure of the state transition model, according to progress of the learning.

27. A method comprising:

receiving time series data for learning;

storing a state transition model for modeling the time series data, the state transition model including state information and state transition information;

initializing structure of the state transition model to sparse structure;

estimating a parameter of the state transition model using the time series data; and

adjusting the structure of the state transition model by, with reference to a first state included in structure of the state transition model and set as a target, removing a second merging state, in which the second merging state is included in the structure of the state transition model, from the structure of the state transition model,

wherein the receiving, the storing, the initializing, the estimating and the adjusting are by a processing unit.

28. A non-transitory storage medium on which is recorded a program executable by a computer, the program comprising:

receiving time series data for learning;

storing a state transition model for modeling the time series data, the state transition model including state information and state transition information;

initializing structure of the state transition model to sparse structure;

estimating a parameter of the state transition model using the time series data; and

adjusting the structure of the state transition model by, with reference to a first state included in the structure of the state transition model and set as a target, removing a second merging state, in which the second merging state is included in the structure of the state transition model, from the structure of the state transition model.

29. An information processing apparatus comprising:

at least one processing unit to;

receive time series data for learning;

store a state transition model for modeling the time series data, the state transition model including state information and state transition information;

estimate a parameter of the state transition model using the time series data; and

adjust structure of the state transition model under conditions of sparse structure,

wherein the structure of the state transition model is adjusted by, with reference to a first state included in the structure of the state transition model and set as a target, removing a second merging state, in which the second merging state is included in the structure of the state transition model, from the structure of the state transition model.

30. The information processing apparatus of claim 29, wherein the structure is adjusted by at least limiting a transition of the state transition model.