Information processing apparatus, method, and program using recurrent neural networks

• US 7,877,338 B2
• Filed: 05/14/2007
• Issued: 01/25/2011
• Est. Priority Date: 05/15/2006
• Status: Expired due to Fees

• Current Assignee: Riken, Sony Corporation (Sony Group Corp.)
• Original Assignee: Riken, Sony Corporation (Sony Group Corp.)
• Inventors: Tani, Jun, Nishimoto, Ryunosuke, Ito, Masato
• Primary Examiner(s): Holmes; Michael B.
• Assistant Examiner(s): Bharadwaj; Kalpana
• Application Number:
  US11/803,237
• Publication Number:
  US 20070265841A1
• Time in Patent Office: 1,352 Days
• Field of Search: 706/15, 700/31, 703/9
• US Class Current: 706/15
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

An information processing apparatus, includes a lower time series data generation unit having a plurality of recurrent neural networks which learn predetermined time series data, and generating prediction time series data. An upper time series data generation unit has recurrent neural networks which learn error time series data that is time series data of errors raised at the time of the learning by the respective plural recurrent neural networks of the lower time series data generation unit. Generation of prediction error time series data that is time series data of prediction errors; and a conversion unit that performs nonlinear conversion for the prediction errors generated by the upper time series data generation unit. The lower time series data generation unit outputs the prediction time series data according to the prediction errors.

Claims

1. An information processing apparatus, including:

lower time series data generation means having a plurality of recurrent neural networks which learn predetermined time series data, and generate prediction time series data according to the learning result;

upper time series data generation means having recurrent neural networks which learn error time series data that is time series data of errors raised at the time of the learning by the respective plural recurrent neural networks of the lower time series data generation means, and generate prediction error time series data that is time series data of prediction errors according to the learning result; and

conversion means for performing nonlinear conversion for the prediction errors generated by the upper time series data generation means,

wherein the lower time series data generation means outputs the prediction time series data generated by the respective plural recurrent neural networks according to the prediction errors which have undergone the nonlinear conversion by the conversion means,

wherein the lower time series data generation means further comprises a plurality of gate means for opening and closing the outputs of the prediction time series data at the subsequent stages of the respective plural recurrent neural networks, and the plural gate means open and close the outputs of the prediction time series data according to the prediction errors which have undergone the nonlinear conversion by the conversion means,

wherein the lower time series data generation means further comprises composition means for compositing and outputting the prediction time series data output from the plural gate means, and

wherein a larger prediction error results a smaller opening of an associated gate means.

2. The information processing apparatus according to claim 1, wherein the recurrent neural networks of the upper time series data generation means are recurrent neural networks of the continuous time type.

3. The information processing apparatus according to claim 1, wherein the conversion means performs the nonlinear conversion for the prediction errors generated by the upper time series data generation means using the softmax function.

4. The information processing apparatus according to claim 1, further comprising temporal filter means for performing the temporal filter processing for the errors output by the lower time series data generation means.

5. The information processing apparatus according to claim 1, further comprising nonlinear filter means for nonlinearly converting the errors output by the lower time series data generation means.

6. The information processing apparatus according to claim 1, wherein, at the time of the learning, the lower time series data generation means updates the weight of learning of the respective plural recurrent neural networks according to errors raised at the time of the learning by the respective plural recurrent neural networks.

7. The information processing apparatus according to claim 1, wherein, at the time of the learning, of errors raised at the time of the learning by the respective plural recurrent neural networks, the lower time series data generation means sets a recurrent neural network that has raised a minimum error to the winner, and updates the weight of learning of the respective plural recurrent neural networks according to the distance from the winner.

8. An information processing method, comprising the steps of:

learning predetermined time series data, and generating prediction time series data according to the learning result;

learning error time series data that is time series data of errors raised at the time of learning the predetermined time series data; and

generating prediction error time series data that is time series data of prediction errors according to the learning result;

performing nonlinear conversion for the generated prediction errors; and

outputting the generated prediction time series data according to the prediction errors which have undergone the nonlinear conversion.

wherein the time series data generation step further comprises a plurality of gate steps for opening and closing the outputs of the prediction time series data at the subsequent stages, and the plural gate open and close the outputs of the prediction time series data according to the prediction errors which have undergone the nonlinear conversion by the conversion step,

wherein the time series data generation step further comprises composition step for compositing and outputting the prediction time series data output from the plural gate steps, and

wherein a large prediction error results a smaller opening of an associated gate step,

9. A non -transitory computer storage medium storing a program that makes a computer execute a processing, the processing comprising the steps of:

learning predetermined time series data, and generating prediction time series data according to the learning result;

learning error time series data that is time series data of errors raised at the time of learning the predetermined time series data, and generating prediction error time series data that is time series data of prediction errors according to the learning, and

performing nonlinear conversion for the generated prediction errors,

wherein the time series data generation step further comprises a plurality of gate steps for opening and closing the outputs of the prediction time series data at the subsequent stages, and the plural gate step open and close the outputs of the prediction time series data according to the prediction errors which have undergone the nonlinear conversion by the conversion step,

wherein the time series data generation step further comprises composition step for compositing and outputting, the prediction time series data output from the plural gate steps, and

wherein a larger prediction error results a smaller opening of an associated gate step.