×

DISCRIMINATIVE PRETRAINING OF DEEP NEURAL NETWORKS

  • US 20130138436A1
  • Filed: 11/26/2011
  • Published: 05/30/2013
  • Est. Priority Date: 11/26/2011
  • Status: Active Grant
First Claim
Patent Images

1. A computer-implemented process for pretraining deep neural network (DNN), comprising:

  • using a computer to perform the following process actions;

    (a) training a single hidden layer neural network (NN) comprising an input layer into which training data is input, an output layer from which an output is generated, and a first hidden layer which is interconnected with the input and output layers with randomly initialized weights, wherein said training comprises,accessing a set of training data entries, each data entry of which has a corresponding label assigned thereto,inputting each data entry of said set one by one into the input layer until all the data entries have been input at least once to produce an initial NN, such that after the inputting of each data entry, said weights associated with the first hidden layer are set via an error BP procedure so that the output generated from the output layer matches the label associated with the training data entry;

    (b) discarding the current output layer and adding a new hidden layer which is interconnected with the last previously trained hidden layer and a new output layer with randomly initialized weights to produce a new multiple hidden layer deep neural network;

    (c) inputting each data entry of said set one by one into the input layer until all the data entries have been input at least once to produce a revised multiple hidden layer deep neural network, such that after the inputting of each data entry, said weights associated with the new hidden layer and each previously trained hidden layer are set via the error BP procedure to produce an output from the new output layer that matches the label associated with the training data entry;

    (d) repeating actions (b) and (c) until a prescribed number of hidden layers have been added; and

    (e) designating the last produced revised multiple layer DNN to be said pretrained DNN.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×