METHOD AND APPARATUS FOR MACHINE LEARNING

• US 20170206450A1
• Filed: 12/19/2016
• Published: 07/20/2017
• Est. Priority Date: 01/18/2016
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: UMEDA, Yuhei
• Granted Patent: US 10,635,975 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

A disclosed machine learning method includes: calculating a first output error between a label and an output in a case where dropout in which values are replaced with 0 is executed for a last layer of a first channel among plural channels in a parallel neural network; calculating a second output error between the label and an output in a case where, the dropout is not executed for the last layer of the first channel; and identifying at least one channel from the plural channels based on a difference between the first output error and the second output error to update parameters of the identified channel.

Claims

1. A non-transitory computer-readable storage medium storing a program that causes a computer to execute a process, the process comprising:

calculating a first output error between a label and an output in a case where dropout in which values are replaced with 0 is executed for a last layer of a first channel among a plurality of channels in a parallel neural network;

calculating a second output error between the label and an output in a case where the dropout is not executed for the last layer of the first channel; and

identifying at least one channel from the plurality of channels based on a difference between the first output error and the second output error to update parameters of the identified channel.

2. The non-transitory computer-readable storage medium as set forth in claim 1, further comprising:

executing the dropout for at least one of values of an output layer in the parallel neural network.

3. The non-transitory computer-readable storage medium as set forth in claim 1, wherein the identifying comprises:

first updating parameters of a second channel that is a channel other than the first channel, upon detecting that a first value obtained by subtracting the second output error from the first output error is greater than a threshold;

second updating parameters of the first channel and the second channel, upon detecting that an absolute value of the first value is smaller than the threshold; and

third updating parameters of the first channel, upon detecting that a second value obtained by subtracting the first output error from the second output error is greater than the threshold.

4. The non-transitory computer-readable storage medium as set forth in claim 3, wherein the second updating comprises:

calculating a first error that is an error of a parameter for calculating values of an output layer from values of the last layer of the first channel and values of a last layer of the second channel, from the first output error, the values of the last layer of the first channel and the values of the last layer of the second channel;

updating, based on the first error, parameters of the second channel by backpropagation;

calculating a second error that is an error of a parameter for calculating values of the output layer from the values of the last layer of the first channel and the values of the last layer of the second channel, from random values in a range calculated by a predetermined method, the values of the last layer of the first channel and the values of the last layer of the second channel; and

updating, based on the second error, parameters of the first channel by the backpropagation.

5. The non-transitory computer-readable storage medium as set forth in claim 3, wherein the second updating comprises:

calculating a first error that is an error of a parameter for calculating values of an output layer from values of the last layer of the first channel and values of a last layer of the second channel, from the first output error, the values of the last layer of the first channel and the values of the last layer of the second channel;

updating, based on the first error, parameters of the second channel by backpropagation; and

replacing parameters of the first channel and a parameter for calculating the values of the output layer from the values of the last layer of the first channel with random values.

6. The non-transitory computer-readable storage medium as set forth in claim 3, wherein the second updating comprises:

calculating a first error that is an error of a parameter for calculating values of an output layer from values of the last layer of the first channel and values of a last layer of the second channel, from the first output error, the values of the last layer of the first channel and the values of the last layer of the second channel;

updating, based on the first error, parameters of the second channel by backpropagation;

replacing an error of a parameter for calculating the values of the output layer from the values of the last layer of the first channel and errors of parameters of the first channel with random values; and

updating the parameter for calculating the values of the output layer from the values of the last layer of the first channel and the parameters of the first channel, by using the replaced error of parameters for calculating the values of the output layer from the values of the last layer of the first channel and the replaced errors of parameters of the first channel.

7. The non-transitory computer-readable storage medium as set forth in claim 3, wherein the third updating comprises:

calculating a first error of a parameter for calculating values of an output layer from values of the last layer of the first channel and values of a last layer of the second channel for which the dropout has been executed, from the second output error, the values of the last layer of the first channel and the values of the last layer of the second channel; and

updating, based on the first error, parameters of the first channel by backpropagation.

8. The non-transitory computer-readable storage medium as set forth in claim 3, wherein the third updating comprises:

keeping parameters of the first channel unchanged, and the process further comprising;

calculating an output in a case where the dropout is not executed for the last layer of the first channel and the dropout is executed for a last layer of a second channel.

9. The non-transitory computer-readable storage medium as set forth in claim 3, wherein the third updating comprises:

keeping parameters of the first channel unchanged, and the process further comprising;

starting processing for a next input.

10. The non-transitory computer-readable storage medium as set forth in claim 3, wherein the first updating comprises:

calculating a first error that is an error of a parameter for calculating values of an output layer from values of the last layer of the first channel and values of a last layer of the second channel from the first output error, the values of the last layer of the first channel and the values of the last layer of the second channel; and

updating, based on the first error, parameters of the second channel by backpropagation.

11. A machine learning method, comprising:

calculating, by using a computer, a first output error between a label and an output in a case where dropout in which values are replaced with 0 is executed for a last layer of a first channel among a plurality of channels in a parallel neural network;

calculating, by using the computer, a second output error between the label and an output in a case where the dropout is not executed for the last layer of the first channel; and

identifying, by using the computer, at least one channel from the plurality of channels based on a difference between the first output error and the second output error to update parameters of the identified channel.

12. An information processing apparatus, comprising:

a memory; and

a processor coupled to the memory and configured to;

calculate a first output error between a label and an output in a case where dropout in which values are replaced with 0 is executed for a last layer of a first channel among a plurality of channels in a parallel neural network;

calculate a second output error between the label and an output in a case where the dropout is not executed for the last layer of the first channel; and

identify at least one channel from the plurality of channels based on a difference between the first output error and the second output error to update parameters of the identified channel.