Method and apparatus for training a neural network

• US 5,664,067 A
• Filed: 01/11/1995
• Issued: 09/02/1997
• Est. Priority Date: 06/19/1992
• Status: Expired due to Term

• Current Assignee: United Parcel Service Of America Incorporated (United Parcel Service Incorporated)
• Original Assignee: United Parcel Service Of America Incorporated (United Parcel Service Incorporated)
• Inventors: Moed, Michael C., Lee, Chih-Ping
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/371,519
• Time in Patent Office: 965 Days
• Field of Search: 395/23, 395/24
• US Class Current: 706/25
• CPC Class Codes: G06F 18/23213 with fixed number of clusters, e.g. K-means clustering
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06V 30/19107 Clustering techniques
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

A plurality of training inputs are selected, wherein each training input corresponds to a first possible output. The quality of each of the plurality of training inputs is characterized. A first training input is selected from the training inputs, where the first training input is of higher quality than a second training input of the training inputs. The neural network is trained with the higher-quality first training input prior to training with the second training input. A neuron may be added to the neural network in accordance with the first training input, wherein the neuron is associated with the first possible output.

Claims

1. A method for training a neural network, comprising the steps of:

(a) organizing one or more sets of training inputs based on quality of the training inputs such that the neural network is trained with higher quality training inputs of each set before being trained with lower quality training inputs of the set, wherein each set of training inputs corresponds to one output; and

(b) sequentially training the neural network using said organized sets of training inputs.

2. The method of claim 1, wherein step (b) comprises the step of training the neural network with the higher quality training inputs of all sets before training the neural network with the lower quality training inputs of any set.

3. The method of claim 1, wherein step (b) comprises the step of training the neural network with all of the training inputs of each set before training the neural network with any of the training inputs of another set.

4. The method of claim 1, wherein:

each training input comprises one or more pixels; and

step (a) comprises the step of basing the quality of the training inputs on a number of pixels that differ between each training input and a corresponding nominal input.

5. The method of claim 1, wherein step (a) comprises the step of basing the quality of the training inputs on a feature measure.

6. The method of claim 5, wherein the feature measure is a Grid feature measure or a Hadamard feature measure.

7. The method of claim 1, wherein step (b) comprises the step of training the neural network with nominal training inputs of each set before training the neural network with degraded training inputs of the set.

8. The method of claim 7, wherein step (b) further comprises the step of training the neural network with the less highly degraded training inputs before training the neural network with more highly degraded training inputs.

9. The method of claim 1, wherein the training inputs are representative of characters.

10. The method of claim 1, wherein the neural network accurately classifies each of the training inputs.

11. An apparatus for training a neural network, comprising:

(a) means for organizing one or more sets of training inputs based on quality of the training inputs such that the neural network is trained with higher quality training inputs of each set before being trained with lower quality training inputs of the set, wherein each set of training inputs corresponds to one output; and

(b) means for sequentially training the neural network using said organized sets of training inputs.

12. The apparatus of claim 11, wherein said means for sequentially training the neural network trains the neural network with the higher quality training inputs of all sets before training the neural network with the lower quality training inputs of any set.

13. The apparatus of claim 11, wherein said means for sequentially training the neural network trains the neural network with all of the training inputs of each set before training the neural network with any of the training inputs of another set.

14. The apparatus of claim 11, wherein:

each training input comprises one or more pixels; and

said means for organizing bases the quality of the training inputs on a number of pixels that differ between each training input and a corresponding nominal input.

15. The apparatus of claim 11, wherein said means for organizing comprises the step of basing the quality of the training inputs on a feature measure.

16. A method for training a neural network, comprising the steps of:

(a) organizing one or more sets of training inputs based on quality of the training inputs, wherein;

each set of training inputs corresponds to one output;

each training input comprises one or more pixels; and

the quality of the training inputs is based on a number of pixels that differ between each training input and a corresponding nominal input; and

(b) sequentially training the neural network using said organized sets of training inputs.

17. A method for training a neural network, comprising the steps of:

(a) organizing one or more sets of training inputs based on quality of the training inputs, wherein each set of training inputs corresponds to one output and the quality of the training inputs is based on a feature measure; and

(b) sequentially training the neural network using said organized sets of training inputs.

18. The method of claim 17, wherein the feature measure is a Grid feature measure or a Hadamard feature measure.

19. A method for training a neural network, comprising the steps of:

(a) organizing one or more sets of training inputs based on quality of the training inputs, wherein each set of training inputs corresponds to one output; and

(b) sequentially training the neural network using said organized sets of training inputs, wherein the neural network is trained with nominal training inputs of each set before training the neural network with degraded training inputs of the set.

20. The method of claim 19, wherein step (b) further comprises the step of training the neural network with the less highly degraded training inputs before training the neural network with more highly degraded training inputs.

21. A method for training a neural network, comprising the steps of:

(a) organizing one or more sets of training inputs based on quality of the training inputs, wherein each set of training inputs corresponds to one output and the training inputs are representative of characters; and

(b) sequentially training the neural network using said organized sets of training inputs.

22. A method for training a neural network, comprising the steps of:

(a) organizing one or more sets of training inputs based on quality of the training inputs, wherein each set of training inputs corresponds to one output; and

(b) sequentially training the neural network using said organized sets of training inputs, wherein the neural network accurately classifies each of the training inputs.