APPARATUS AND METHOD FOR TRAINING DEEP LEARNING MODEL

• US 20200134454A1
• Filed: 10/28/2019
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Abandoned Application

• Current Assignee: Samsung SDS (China) Co., Ltd. (Samsung Group)
• Original Assignee: Samsung SDS (China) Co., Ltd. (Samsung Group)
• Inventors: Choi, Jong-Won, Kim, Ji-Hoon, Choi, Young-Joon
• Application Number:
  US16/665,751
• Publication Number:
  US 20200134454A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

An apparatus and a method for training a deep learning model are disclosed. According to the disclosed embodiments, performance of deep learning can be enhanced by performing bidirectional training of learning information on the target dataset on the basis of the source dataset and learning information on the source dataset on the basis of the target dataset.

Claims

1. A method for training deep learning model, which is performed by a computing device comprising one or more processors and a memory for storing one or more programs executed by the one or more processors, the method comprising:

training a deep learning-based forward network using a source dataset assigned with a first label and a target dataset not assigned with a label as training data;

determining a final noisy label matrix for a deep learning-based inverse network using a plurality of previously generated noisy label matrixes and the inverse network;

training the inverse network on the basis of the final noisy label matrix;

training a deep learning-based integrated network, which is combined the trained forward network and the trained inverse network, on the basis of the first label and a second label of the source dataset; and

determining the forward network included in the trained integrated network as a deep learning model.

2. The method according to claim 1, wherein the training of the forward network comprises training the forward network to extract a label for the target dataset not assigned with a label on the basis of the source dataset assigned with the first label, wherein the forward network is trained on the basis of a loss function set in the forward network.

3. The method according to claim 1, further comprising extracting a third label for the target dataset not assigned with a label from the source dataset assigned with the first label and the target dataset not assigned with a label using the trained forward network.

4. The method according to claim 3, wherein the determining of the final noisy label matrix comprises:

training the inverse network for the first time to extract a label for the source dataset not assigned with a label on the basis of the target dataset assigned with the third label, wherein the inverse network is trained on the basis of a loss function set in the inverse network;

retraining the first-time trained inverse network on the basis of each of the plurality of noisy label matrixes;

extracting a fourth label for the source dataset not assigned with a label from the target dataset assigned with the third label and the source dataset not assigned with a label using the trained inverse network on the basis of the each of the plurality of noisy label matrixes; and

determining one of the plurality of noisy label matrixes as the final noisy label matrix on the basis of the first label and the fourth label.

5. The method according to claim 4, wherein the training of the inverse network comprises retraining the first-time trained inverse network on the basis of the final noisy label matrix and a loss function based on cross entropy.

6. The method according to claim 3, wherein the training of the inverse network comprises extracting the second label for the source dataset not assigned with a label from the target dataset assigned with the third label and the source dataset not assigned with a label using the trained inverse network, and the training of the integrated network comprises training the integrated network on the basis of the loss function set in the integrated network so that the value of the second label may approach the value of the first label.

7. The method according to claim 6, wherein the loss function set in the integrated network is generated on the basis of loss functions respectively set in the forward network and the inverse network and a loss function based on an auto-encoder.

8. The method according to claim 1, wherein the training of the integrated network comprises calculating accuracy of each of a plurality of training samples included in the source dataset and the target dataset on the basis of the final noisy label matrix, and training the integrated network using a training sample of which the accuracy is higher than a preset value among the plurality of training samples.

9. An apparatus for training deep learning model comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors and comprise instructions for executing the steps of:

training a deep learning-based forward network using a source dataset assigned with a first label and a target dataset not assigned with a label as training data;

determining a final noisy label matrix for a deep learning-based inverse network using a plurality of previously generated noisy label matrixes and the inverse network;

training the inverse network on the basis of the final noisy label matrix;

training a deep learning-based integrated network, which is combined the trained forward network and the trained inverse network, on the basis of the first label and a second label of the source dataset; and

determining the forward network included in the trained integrated network as a deep learning model.

10. The apparatus according to claim 9, wherein the step of training the forward network trains the forward network to extract a label for the target dataset not assigned with a label on the basis of the source dataset assigned with the first label, wherein the forward network is trained on the basis of a loss function set in the forward network.

11. The apparatus according to claim 9, wherein the one or more programs further comprise instructions for executing the step of extracting a third label for the target dataset not assigned with a label from the source dataset assigned with the first label and the target dataset not assigned with a label using the trained forward network.

12. The apparatus according to claim 11, wherein the step of determining a final noisy label matrix includes the steps of:

training the inverse network for the first time to extract a label for the source dataset not assigned with a label on the basis of the target dataset assigned with the third label, wherein the inverse network is trained on the basis of a loss function set in the inverse network;

retraining the first-time trained inverse network on the basis of each of the plurality of noisy label matrixes;

extracting a fourth label for the source dataset not assigned with a label from the target dataset assigned with the third label and the source dataset not assigned with a label using the trained inverse network on the basis of the each of the plurality of noisy label matrixes; and

determining one of the plurality of noisy label matrixes as the final noisy label matrix on the basis of the first label and the fourth label.

13. The apparatus according to claim 12, wherein the step of training the inverse network comprises the step of retraining the first-time trained inverse network on the basis of the final noisy label matrix and a loss function based on cross entropy.

14. The apparatus according to claim 11, wherein the step of training the inverse network extracts the second label for the source dataset not assigned with a label from the target dataset assigned with the third label and the source dataset not assigned with a label using the trained inverse network, and the step of training an integrated network trains the integrated network on the basis of the loss function set in the integrated network so that the value of the second label may approach the value of the first label.

15. The apparatus according to claim 14, wherein the loss function set in the integrated network is generated on the basis of loss functions respectively set in the forward network and the inverse network and a loss function based on an auto-encoder.

16. The apparatus according to claim 9, wherein the step of training the integrated network calculates accuracy of each of a plurality of training samples included in the source dataset and the target dataset on the basis of the final noisy label matrix, and trains the integrated network using a training sample of which the accuracy is higher than a preset value among the plurality of training samples.