Time-series trend estimating system and method using column-structured recurrent neural network

• US 5,956,702 A
• Filed: 08/22/1996
• Issued: 09/21/1999
• Est. Priority Date: 09/06/1995
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Golea, Mostefa, Matsuoka, Masahiro
• Primary Examiner(s): Stamber, Eric W.
• Assistant Examiner(s): Patel, Jagdish
• Application Number:
  US08/701,700
• Time in Patent Office: 1,125 Days
• Field of Search: 395/22, 395/24, 395/27, 704/250, 706/20, 706/22, 706/19, 706/34, 706/25
• US Class Current: 706/22
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

Each neural element of a column-structured recurrent neural network generates an output from input data and recurrent data provided from a context layer of a corresponding column. One or more candidates for an estimated value is obtained, and an occurrence probability is computed using an internal state by solving an estimation equation determined by the internal state output from the neural network. A candidate having the highest occurrence probability is an estimated value for unknown data. Thus, the internal state of the recurrent neural network is explicitly associated with the estimated value for data, and a data change can be efficiently estimated.

Claims

1. A time-series trend estimating system for use in an information processing apparatus for estimating a change trend of data which changes with time, comprising:

input means for inputting time-series data;

neural network means for outputting an internal state including information about past time-series data, comprising a column-structured recurrent neural network including at least one independent column with a context layer;

estimated value generating means for obtaining an occurrence probability of a candidate for an estimated value according to the internal state, and obtaining a most probably candidate as the estimated value; and

output means for outputting the estimated value as an estimation result of unknown data.

2. The time-series trend estimating system according to claim 1, wherein

said neural network means is adapted for outputting the internal state representing information about a probability that a probability distribution of data is selected.

3. The time-series trend estimating system according to claim 1 further comprising:

learning means for learning provided time-series data, generating an optimum value of a parameter of said recurrent neural network means, and for providing the optimum value for said neural network means, wherein

said neural network means is adapted for computing the internal state using the optimum value of the parameter.

4. The time-series trend estimating system according to claim 3, wherein

said learning means is adapted for computing an evaluation function using an arbitrary value of the parameter of said recurrent neural network and the provided time-series data, and setting as the optimum value a parameter value causing a value of the evaluation function to be optimal.

5. The time-series trend estimating system according to claim 1, wherein

said column-structured recurrent neural network comprises a plurality of columns, each of said columns including:

a neural element for receiving said time-series data; and

a shift register in the context layer for holding a past output history of the neural element in time order,

and an output of the shift register is recurrently input to the neural element in a same column.

6. The time-series trend estimating system according to claim 1, wherein

said column-structured recurrent neural network comprises a plurality of columns, each of said columns including:

a neural element for receiving said time-series data; and

a register in the context layer for holding a past output history of the neural element,

and an output of the register is recurrently input to the neural element in a same column.

7. The time-series trend estimating system according to claim 6, wherein

said estimated value generating means is adapted for assigning to said neural element a discrete variable representing a discreetness of a discontinuous trend of time-series data, generating a probability distribution function defined by an energy function represented by the discrete variable and the internal state, and obtaining the estimated value using the probability distribution function.

8. The time-series trend estimating system according to claim 7, wherein

said estimating means is adapted for generating the probability distribution function as a combination of a plurality of Gaussian distribution functions with a probability distribution of the discrete variable used as a weight.

9. The time-series trend estimating system according to claim 1, wherein

said estimating means is adapted for generating a probability distribution function as a combination of a plurality of Gaussian distribution functions using the internal state, and obtaining the estimated value using the probability distribution function.

10. The time-series trend estimating system according to claim 9, wherein

said estimating means is adapted for generating a string of probability distribution functions, each of the probability distribution functions corresponding to a form of said probability function which changes with time through update of the internal state, and generating a string of estimated values string, each of the estimated values corresponding to a value of said estimated value which changes with time, from the string of the probability distribution functions.

11. The time-series trend estimating system according to claim 1, wherein

said estimated value generating means is adapted for setting a solution of an estimation equation determined by the internal state and a parameter of the recurrent neural network as the candidate for the estimated value.

12. The time-series trend estimating system according to claim 11, wherein

said estimating means comprises zero point computing means for obtaining a zero point of an object function described with the internal state and the parameter of the recurrent neural network, and is adapted for setting a data value corresponding to the zero point as the candidate for the estimated value.

13. The time-series trend estimating system according to claim 12, wherein

said estimating means further comprises object function evaluating means for computing a value of the object function around the zero point from the internal state, and providing a computation result for the zero point computing means.

14. The time-series trend estimating system according to claim 11, wherein

said estimating means further comprises probability computing means for computing a value of a probability distribution function of data according to the candidate for the estimated value and internal state.

15. The time-series trend estimating system according to claim 14, wherein

said recurrent neural network comprises a plurality of columns having a neural element;

said estimating means further comprises normalization coefficient computing means for computing a normalization coefficient of the probability distribution function using a discrete variable assigned to the neural element; and

said probability computing means is adapted for computing the value of the probability distribution function using the normalization coefficient obtained by said normalization coefficient computing means.

16. The time-series trend estimating system according to claim 14 further comprising:

estimated value selecting means for providing a plurality of candidates for the estimated value for said probability computing means, receiving a plurality of values of the corresponding probability distribution function, and selecting a candidate corresponding to a maximum value in the plurality of values as the estimated value.

17. A time-series trend estimating system comprising:

input means for receiving time-series data which changes with time;

neural network means for storing an internal state representing information about a probability in which a probability distribution relating to a discrete value of the time-series data is selected; and

estimated value output means for obtaining an estimated value having a high occurrence probability using the internal state, and outputting the estimated value as an estimation result.

18. A computer-readable storage medium used by a computer for estimating a change trend of data which changes with time, and used to direct the computer to perform the steps of:

inputting time-series data;

obtaining an internal state including information about past time-series data using a column-structured recurrent neural network having at least one independent column with a context layer;

obtaining an occurrence probability of a candidate for an estimated value using the internal state, and obtaining a candidate having a highest probability as the estimated value; and

estimating unknown data based on the estimated value.

19. The computer-readable storage medium according to claim 18 used to direct the computer to further perform the step of:

outputting the internal state representing information about a probability that a probability distribution of the data is selected.

20. A time-series trend estimating method of estimating a change trend of data which changes with time, comprising the steps of:

measuring time-series data;

obtaining an internal state including information about past time-series data through a column-structured recurrent neural network having at least one independent column with a context layer;

obtaining an occurrence probability of a candidate for an estimated value using the internal state;

setting a candidate having a highest probability as the estimated value; and

estimating unknown data based on the estimated value.

21. The time-series trend estimating method according to claim 20, wherein

said internal state represents information about a probability that a probability distribution of data is selected.