Neural network, neuron, and method for recognizing a missing input valve

• US 5,448,684 A
• Filed: 11/12/1993
• Issued: 09/05/1995
• Est. Priority Date: 11/12/1993
• Status: Expired due to Fees

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Holt, James C.
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): HAFIZ, TARIQ R
• Application Number:
  US08/150,295
• Time in Patent Office: 662 Days
• Field of Search: 395/23, 395/24, 395/27, 395/22-25, 382/12-15
• US Class Current: 706/20
• CPC Class Codes: G06N 3/0463 Neocognitrons

Abstract

A neuron (100) has a null-inhibiting function so that null inputs do not affect the output of the neuron (100) or updating of its weights. The neuron (100) provides a net value based on a sum of products of each of several inputs, and corresponding weight and null values, and provides an output in response to the net value. A neural network (40) which uses such a neuron (100) has a first segmented layer (41) in which each segment (50-52) corresponds to a manufacturing process step (60-62). Each segment of the first layer (41) receives as inputs measured values associated with the process step (60-62). A second layer (42) connected to the first layer (4l), is non-segmented to model the entire manufacturing process (80). The first (41) and second (42) layers are both unsupervised and competitive. A third layer (43) connected to the second layer (42) then estimates parameters of the manufacturing process (80) and is unsupervised and noncompetitive.

Claims

1. A neuron for recognizing a missing input value, comprising:

a predetermined number of first inputs for receiving data inputs of the neuron;

a plurality of second inputs corresponding to each of said predetermined number of first inputs for receiving null values;

each null value being in either a first logic state if a corresponding data input is missing, or a second logic state otherwise;

a plurality of nullable weighting elements coupled to said predetermined number of first inputs and to said plurality of second inputs, each nullable weighting element having a weight associated therewith and providing an output thereof as a product of either a corresponding data input and said weight when a corresponding null value is in said first logic state, or as a predetermined value otherwise;

a summing device coupled to said plurality of nullable weighting elements, for summing outputs of each nullable weighting element to provide a net output; and

an activation element coupled to said summing device, for providing an output of the neuron in response to said net output and a predetermined activation function.

2. The neuron of claim 1 wherein said predetermined activation function is a Kohonen function.

3. The neuron of claim 1 wherein each of said plurality of nullable weighting elements updates said weight thereof in response to said corresponding data input and said corresponding null value.

4. The neuron of claim 3 wherein each of said plurality of nullable weighting elements updates said weight thereof equal to a prior weight, plus a product of, a training rate coefficient, said corresponding null value, and a difference between said corresponding data input and said prior weight.

5. The neuron of claim 1 further comprising an inhibit detector for activating an inhibit output signal in response to said corresponding null of each said plurality of nullable weighting elements being active.

6. The neuron of claim 1 wherein said activation element further has a plurality of net inputs and provides said output of the neuron further in response to said plurality of net inputs.

7. The neuron of claim 6 wherein said activation element further has an input for receiving an absolute null signal, and provides said output of the neuron in an inactive state further in response to said absolute null signal being active.

8. The neuron of claim 1 wherein said predetermined activation function is a Grossberg function.

9. The neuron of claim 1 further comprising a learning element for adjusting said weight associated with each of nullable said plurality of weighting elements in response to said output of the neuron and a training information signal.

10. A neuron for recognizing a missing input value, comprising:

a predetermined number of first inputs for receiving inputs x_i of the neuron;

a plurality of second inputs corresponding to each of said predetermined number of first inputs for receiving null values n_i ;

each null value n_i being in either a first logic state if a corresponding input x_i is missing, or a second logic state otherwise;

a plurality of null-inhibiting weighting elements for storing a plurality of weights w_i corresponding to each of said predetermined number of first inputs, each null-inhibiting weighting element providing an output thereof y'"'"'_i as a product of a corresponding input x_i, said a corresponding weight w_i, and a corresponding null value n_i ;

a summing device coupled to said plurality of null-inhibiting weighting elements, for summing outputs of each null-inhibiting weighting element to provide a net output NET defined by NET =Σ

x_i *w_i *n_i ; and

an activation element coupled to said summing device, for providing an output of the neuron in response to said net output NET and a predetermined activation function.

11. The neuron of claim 10 wherein said predetermined activation function is a Kohonen function.

12. The neuron of claim 10 wherein each of said plurality of null-inhibiting weighting elements updates a weight thereof in response to said corresponding input x_i and said corresponding null value n_i.

13. The neuron of claim 12 wherein each of said plurality of null-inhibiting weighting elements updates said weight thereof w_i equal to a prior weight, plus a product of, a training rate coefficient, said corresponding null value n_i, and a difference between said corresponding input x_i and said prior weight.

14. The neutron of claim 10 further comprising an inhibit detector for activating an inhibit output signal in response to each corresponding null value n_i being active.

15. The neuron of claim 10 wherein said activation element further has a plurality of net inputs and provides said output of the neuron further in response to said plurality of net inputs.

16. The neuron of claim 15 wherein said activation element further has an input for receiving an absolute null signal, and provides said output of the neuron in an inactive state further in response to said absolute null signal being active.

17. The neuron of claim 10 wherein said predetermined activation function is a Grossberg function.

18. The neuron of claim 10 further comprising a learning element for adjusting said corresponding weight w_i of each of said plurality of null-inhibiting weighting elements in response to said output of the neuron and a training information signal.

19. A neural network for recognizing a missing input value comprising:

a first layer of segmented neurons of a first type receiving a plurality of input signals and a plurality of null inputs corresponding to each of said plurality of input signals, and providing a plurality of first output signals in response to said plurality of input signals and said plurality of null inputs;

a second layer of neurons of a second type operably coupled to said first layer of segmented neurons, said second layer of neurons receiving each of said plurality of first output signals, and activating at least one of a plurality of second output signals in response; and

a third layer of neurons of a third type operably coupled to said second layer of neurons, said third layer of neurons receiving said plurality of second output signals, and providing at least one output of the neural network in response to said at least one of said plurality of second output signals activated by said second layer of neurons;

said first and second types characterized as being unsupervised and competitive;

said third type characterized as being supervised and noncompetitive.

20. The neural network of claim 19, wherein said first layer of segmented neurons comprises a plurality of neurons, each of said plurality of neurons including a plurality of weighting elements which conditionally updates weights thereof in response to a corresponding one of said plurality of null inputs.

21. The neural network of claim 19 wherein said first layer of segmented neurons provides said plurality of first output signals in response to said plurality null inputs.

22. A method for recognizing a missing input value while estimating an unknown parameter, comprising the steps of:

inputting a plurality of first measured values and a plurality of first null values corresponding thereto to a first neuron segment, each of said plurality of first null values indicating when in an active logic state thereof that a corresponding one of said plurality of first measured values is missing;

inputting a plurality of second measured values and a plurality of second null values corresponding thereto to a second neuron segment, each of said plurality of second null values indicating when in an active logic state thereof that a corresponding one of said plurality of second measured values is missing;

operably coupling together each of a plurality of neurons of said first neuron segment such that each of said plurality of neurons of said first neuron segment provides an output thereof in partial dependence on said output of others of said plurality of neurons of said first neuron segment;

operably coupling together each of a plurality of neurons of said second neuron segment such that each of said plurality of neurons of said second neuron segment provides an output thereof in partial dependence on said output of others of said plurality of neurons of said second neuron segment; and

estimating the unknown parameter from said outputs of each of said first and second neuron segments when at least one of said pluralities of first and second null values is inactive.

23. The method of claim 22 wherein said step of estimating the unknown parameter comprises the steps of:

outputting said outputs of each of said first and second neuron segments, forming a first layer of segmented neurons, to a second layer of neurons;

operably coupling together each of a plurality of neurons of said second layer of neurons such that each of said plurality of neurons of said second layer of neurons provides an output thereof in partial dependence on said output of others of said plurality of neurons within said second layer of neurons; and

estimating the unknown parameter in response to a plurality of outputs of said second layer of neurons.

24. The method of claim 23 wherein said step of estimating the unknown parameter comprises the steps of:

outputting said outputs of said second layer of neurons to a third layer of neurons; and

estimating the unknown parameter as an output of said third layer of neurons.

25. The method of claim 24 further comprising the step of training said third layer of neurons in response to a training information input.