Neuro-computer system for executing a plurality of controlling algorithms

• US 5,524,175 A
• Filed: 10/27/1993
• Issued: 06/04/1996
• Est. Priority Date: 10/29/1992
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd., Hitachi Microcomputer System Limited
• Original Assignee: Hitachi, Ltd., Hitachi Microcomputer System Limited
• Inventors: Hashimoto, Masashi, Ochiai, Tatsuo, Sato, Yuji, Takayanagi, Hiroshi, Kuwabara, Yoshihiro, Shibata, Katsunari, Moki, Keiji, Okahashi, Tatsuo, Sakaguchi, Takahiro, Ogata, Hisao, Asai, Mitsuo, Ohki, Masaru
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): DORVIL, RICHEMOND
• Application Number:
  US08/141,798
• Time in Patent Office: 951 Days
• Field of Search: 395/11, 395/22, 395/23, 395/24, 395/27
• US Class Current: 706/41
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/10 Interfaces, programming languages or software development kits, e.g. for simulating neural networks

Abstract

A general neuro-computer and system using it is capable of executing a plurality of learning algorithms, providing an instruction execution speed comparable with a hard wired system, and practically neglecting a time required for rewriting microprograms. The neuro-computer is constituted by a neuron array having a plurality of neurons, a control storage unit for storing microinstructions, a parameter register, a control logic, and a global memory. A host computer as a user interface inputs information necessary for the learning and execution of the neuro-computer to the system, the information including learning algorithms, neural network architecture, the number of learnings, the number of input patterns, input signals, and desired signals. The information inputted from the host computer is transferred via a SCSI to the neuro-computer to perform a desired neural network operation.

Claims

1. A neuro-computer comprising:

a neuron array having a plurality of neurons each including at least the following elements (a) to (l),

(a) a rewritable memory and a register file,

(b) a memory write value holding means for holding data to be written in said rewritable memory,

(c) a shifter and an ALU (Arithmetic Logic Unit),

(d) a signal selecting means for selecting two values, in accordance with a control signal from a control logic unit, from at least one signal including inputs from said rewritable memory, said register, and an external circuit of said neuron, and outputs from said ALU and said shifter,

(e) a select signal holding means for holding said selected two values,

(f) a multiplier for multiplying the values held by said select signal holding means,

(g) a multiplication result holding means for holding a multiplication result of said multiplier,

(h) an operation result holding means for holding an output from said ALU or said shifter,

(i) a shifter input selecting means and an ALU input selecting means for selecting inputs to said shifter and said ALU from outputs from said operation result holding means, said select signal holding means, and said multiplication result holding means,

(j) a first operation result selecting means for selecting outputs from said shifter and said ALU, and determining an input to said operation result holding means,

(k) a second operation result selecting means for selecting the outputs from said shifter and said ALU, and determining an output to said memory write value holding means, said register file, output selecting means for selecting an output to the external circuit of said neuron, and said signal selecting means for selecting said two values, and

(l) said output selecting means for selecting an output to the external circuit of said neuron, from the outputs from said shifter, said ALU, said rewritable memory, and said register file;

a control storage unit for storing a microinstruction for controlling said neuron array;

a global memory unit for storing data necessary for information processing at said neuron array; and

said control logic unit including a means for controlling said control storage unit, a means for controlling said global memory unit, a neuron array controlling means for controlling said neuron array in accordance with said microinstruction in said control storage unit, and an external bus interface means for accessing data via an external bus.

2. A neuro-computer according to claim 1, wherein said neuron array controlling means of said control logic unit includes a circuit for generating an address of said neuron, a circuit for generating said microinstruction for said neuron, a circuit for generating an input signal to said neuron, and a circuit for processing an output signal from said neuron.

3. A neuro-computer according to claim 2, wherein a plurality of neurons are controlled by the same control signal from said circuit, for generating said microinstruction for said neuron of said control logic unit.

4. A neuro-computer according to claim 3, wherein said neuron includes a condition code register for rewriting or holding the operation condition outputted from said shifter or said ALU in accordance with a control signal from said control logic unit, and data write to said rewritable memory or said register file of said neuron is controlled in accordance with a value selected from said condition code register in response to another control signal from said control logic unit.

5. A neuro-computer according to claim 4, wherein said condition code register holds the operation results of said ALU including,

(1) a bit representing a sign,

(2) a bit representing whether an operation result is "0",

(3) a bit representing a carry-out, and

(4) a bit representing an overflow, in accordance with a control signal from said control logic unit.

6. A neuro-computer according to claim 5, wherein said neuron includes a group register for loading the value of said condition code register or an inverted signal of said value in accordance with a control signal from said control logic unit, and data write to said rewritable memory, said register file, or said operation result holding means, respectively in said neuron, is controlled in accordance with the value loaded in said group register.

7. A neuro-computer according to claim 6, wherein said neuron controls the data write to said rewritable memory, said register file, or said operation result holding means, in accordance with an AND (logical product) value between

(1) the value in said group register, and

(2) the value selected from said condition code register in accordance with a control signal from said control logic unit.

8. A neuro-computer according to claim 4, wherein said neuron includes a neuron request register which is written with an AND or OR value between the value in said condition code register or an inverted signal of a value, and the value already written in said neuron request register, or set to "0" or "1", in accordance with a control signal from said control logic unit, and an AND or OR value signal between the values of said neuron request registers of a plurality of neurons is transferred to said control logic unit to control the entirety of said neuro-computer in accordance with a control signal.

9. A neuro-computer according to claim 8, wherein the whole operation of said neuro-computer is intercepted in accordance with said signal outputted from said neurons.

10. A neuro-computer according to claim 3, wherein said neuron is arranged to truncate lower bits excepting a sign bit of a transferred signal if the number of bits of said signal is to be changed before and after said multiplier.

11. A neuro-computer according to claim 3, wherein said neuron includes signal lower bit truncation means or signal right shift means, and if the bit portion of a negative number to be cut off has "1", the number left untruncated is added with "1".

12. A neuro-computer according to claim 3, wherein a number of pipelines present while data is transferred from said global storage unit to said neuron after said microinstruction is issued from said control logic unit, is set to be equal to the number of pipelines present while data is transferred from said neuron to another neuron after said microinstruction is issued from said control logic unit.

13. A neuro-computer according to claim 12, further comprising flip-flops which coincide in number with said number of pipelines.

14. A neuro-computer according to claim 13, wherein said neuro-computer is arranged such that, if the operation of the neural network is intercepted in response to a command from the external circuit or in response to an internal operation state, the operation is intercepted after data having been left in the pipelines when an interception command was issued is saved, and at the following operation start, the saved data is sequentially outputted for the execution of the operation.

15. A neuro-computer according to claim 14, wherein said neuro-computer is arranged such that, of the data in the pipelines at the time of interception, the data still not transferred to said neuron and a control signal are transferred to said neuron, and the output data from said neuron is saved.

16. A neuro-computer according to claim 14, wherein said neuro-computer is arranged such that, if an interception command is again issued after the start and before all the saved data has been outputted, only the data in the pipelines corresponding to the outputted and saved data is saved.

17. A neuro-computer according to claim 1, wherein said neuron is configured such that said multiplier performs a fixed point multiplication by using a predetermined number of bits, and a desired neural network operation is executed by using a weight value, an output value, and a back propagation error, respectively of a predetermined number of bits, at said shifter and said ALU, said neuron including:

a means for storing a weight value having a number of bits greater than said predetermined number of bits at the time of learning of the weight value of said neural network;

a means for reading and operating a predetermined number of bits of each stored weight value, for calculation of a neuron output value and a back propagation error in a learning; and

a means for reading the number of bits greater than said predetermined number of bits, for modification of the weight value.

18. A neuro-computer according to claim 1, wherein said neuron is configured such that said multiplier performs a fixed point multiplication by using a predetermined number of bits, and a desired neural network operation is executed by using a weight value, an output value, and a back propagation error, respectively of a predetermined number of bits, at said shifter and said ALU, said neuron including:

a means for storing a plurality of microinstructions for a learning with back propagation errors having different point positions;

a means for storing a condition of changing said microinstruction for the learning; and

a means for selecting another one of a plurality of microinstructions for the learning if said changing condition is satisfied during the course of the learning.

19. A neuro-computer according to claim 1, wherein said neuron is configured such that said multiplier performs a fixed point multiplication by using a predetermined number of bits, and a desired neural network operation is executed by using a weight value, an output value, and a back propagation error, respectively of a predetermined number of bits, at said shifter and said ALU, said neuron including:

a means for storing a condition of changing a point position of a back propagation error during a learning of a weight value;

a means for changing said point position of a back propagation error if said condition of changing is satisfied during the learning; and

a means for continuing the learning by using a back propagation error with said changed point position.

20. A neuro-computer according to claim 1, wherein said neuron is configured such that said multiplier performs a fixed point multiplication by using a predetermined number of bits, and a desired neural network operation is executed by using a weight value, an output value, and a back propagation error, respectively of a predetermined number of bits, at said shifter and said ALU, said neuron including:

a means for storing a plurality of microinstructions for a learning with back propagation errors having different point positions;

a means for selecting said microinstructions for the learning from said plurality of programs before the start of the learning; and

a means for performing the learning of weight values by using said selected microinstruction.

21. A neuro-computer according to claim 1, wherein said neuron is configured such that said multiplier performs a fixed point multiplication by using a predetermined number of bits, and a desired neural network operation is executed by using a weight value, an output value, and a back propagation error, respectively of a predetermined number of bits, at said shifter and said ALU, said neuron including:

a means for changing a point position of a back propagation error;

a means for selecting one of a plurality of point positions by using said changing means before a start of learning; and

a means for performing a learning of weight values by using back propagation errors with said selected point position.

22. A neuro-computer system comprising:

a neuro-computer processing information by using at least one type of a neural network, said neuro-computer including a neuron array comprising at least one neuron, a control storage unit for storing a microinstruction for controlling said neuron array, a global storage unit for storing data necessary for processing information by using said neuron array, a parameter storage unit for storing parameter information including the number of learnings and the number of input patterns, and a control logic unit for controlling said control storage unit, said global storage unit, and said parameter storage unit by using said microinstruction, said parameter information, and data necessary for a neural network operation;

a host computer for controlling at least a start and stop of said neuro-computer and inputting necessary information for the learning and execution of said neuro-computer to said neuro-computer, said necessary information including a learning algorithm, a neural network architecture, a number of learnings, a number of input patterns, an input signal, and a desired signal (an expected value); and

a means for interconnecting said neuro-computer and said host computer,

wherein said neuron of said neuro-computer includes;

a logic circuit for realizing a neural network operation;

a condition code register for indicating an operation result of said logic circuit;

a group register for holding the value of said condition code register and an inverted value of said value and for controlling the operation and grouping of said neuron;

a neuron request register for holding the value of said condition code register and an inverted value of said value and for supplying said control logic unit with the state of said neuron;

a means for storing, when executing the neural network operation by using a weight value, an output value, and a back propagation error, respectively of a predetermined number of bits, a weight value having a number of bits greater than said predetermined number of bits;

a means for reading and operating a weight value of a predetermined number of bits from said weight value storing means when calculating a neuron output value and a back propagation error during the operation; and

a means for reading and operating a weight value having the number of bits greater than said predetermined number of bits when modifying the weight value during the operation,

said control logic unit of said neuro-computer comprising;

a means for controlling said control storage means;

a means, having an address translation table for generating a neuron address and a logic circuit for detecting the maximum and minimum values of neuron output values, for controlling said neuron array in accordance with said microinstruction read from said control storage unit;

a means for controlling said global memory unit and said parameter storage unit; and

an external bus interface means for accessing data via an external bus,

said neuro-computer comprising;

a first pipeline latch provided on a bus of said neuro-computer;

a second pipeline latch for transferring data from said neuron to another neuron after said microinstruction is issued, said second pipeline latch having a number of stages equal to a number of stages of a pipeline latch for transferring data from said global storage unit to said neuron after said microinstruction is issued;

a register for saving data left in said first and second pipeline latches when the neural network operation is intercepted; and

a means for sequentially outputting said saved data when executing again said intercepted operation, and

said host computer comprising;

a means for generating said microinstruction describing the neural network operation from network definition information including the number of layers of the neural network and the number of neurons at each layer;

a time sequential data receiving means for periodically receiving time sequential data via a communication line;

a data converting means for converting said received time sequential data into a data format capable of being processed by said neuro-computer;

a data transmitting/receiving means for transferring said converted data to said neuro-computer, and receiving data transferred from said neuro-computer; and

a display and processing means for processing and displaying the data received by said data transmitting/receiving means.

23. A neuro-computer system according to claim 22, wherein said time sequential data is at least one of economic fundamental data including stock value data, money rate data, exchange data, and price data, and said neuro-computer predicts said time sequential data or optimizes a combination of said time sequential data.

24. A neuro-computer system comprising:

a neuro-simulator for realizing a neural network operation by using software;

a neuro-computer for realizing the neural network operation by using hardware;

a neuro-learning information generating means for generating neuro-learning information necessary for neural network learning;

a data switching information storage means for storing reference information used for a judgement of switching between said software and said hardware; and

a data switching means for switching said neuro-learning information in accordance with said reference information.

25. A neuro-computer system according to claim 24, wherein said neuro-computer system is arranged such that, said data switching information storage means stores at least a learning performance, precision, and error, respectively, of said neuro-simulator and said neuro-computer, and an overhead time for learning by said neuro-computer.

26. A neuro-computer system according to claim 25, wherein said neuro-computer is arranged such that, said neuro-computer executes a BP (Back Propagation) algorithm which is one of neural network learning algorithms.

27. A neuro-computer system according to claim 25, wherein said neuro-computer is arranged such that, said neuro-computer executes an LVQ (Learning Vector Quantization) algorithm which is one of neural network learning algorithms.

28. A neuro-computer system according to claim 25, wherein said neuro-computer is arranged such that, said neuro-computer executes said LVQ algorithm which is one of neural network learning algorithms, and by using learning results of said LVQ algorithm, continuously executes said BP algorithm which is one of neural network learning algorithms.

29. A neuro-computer system according to claim 25, wherein said neuro-computer is arranged such that, said neuro-computer executes said BP algorithm which is one of neural network learning algorithms, and by using learning results of said BP algorithm, continuously executes said LVQ algorithm which is one of neural network learning algorithms.

30. A neuro-computer system according to claim 25, wherein said neuro-computer is arranged such that, said neuro-computer executes a GA (Generic Algorithm) which is one of massive parallel learning algorithms.