TEACHER AND STUDENT LEARNING FOR CONSTRUCTING MIXED-DOMAIN MODEL

• US 20190378006A1
• Filed: 06/08/2018
• Published: 12/12/2019
• Est. Priority Date: 06/08/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fukuda, Takashi, Ichikawa, Osamu, Thomas, Samuel, Ramabhadran, Bhuvana
• Granted Patent: US 11,416,741 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06F 2218/00 Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/7753 Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training

Abstract

A technique for constructing a model supporting a plurality of domains is disclosed. In the technique, a plurality of teacher models, each of which is specialized for different one of the plurality of the domains, is prepared. A plurality of training data collections, each of which is collected for different one of the plurality of the domains, is obtained. A plurality of soft label sets is generated by inputting each training data in the plurality of the training data collections into corresponding one of the plurality of the teacher models. A student model is trained using the plurality of the soft label sets.

Claims

1. A computer-implemented method for constructing a model supporting a plurality of domains, the method comprising:

preparing a plurality of teacher models, each teacher model being specialized for different one of the plurality of the domains;

obtaining a plurality of training data collections, each of the plurality of training data collections being collected for a different one of the plurality of the domains;

inputting training data from each of the plurality of training data collections into a corresponding one of the plurality of the teacher models to generate a plurality of soft label sets; and

training a student model using the plurality of the soft label sets.

2. The method of claim 1, wherein each teacher model is connected to a matched feature extractor for corresponding one of the plurality of the domains and the student model is connected to a unified feature extractor, the unified feature extractor being common at least partially to the plurality of the domains.

3. The method of claim 2, wherein the matched feature extractor of each teacher model extracts a matched feature from an input signal in the corresponding one of the plurality of the domains and the preparing of the plurality of the teacher models comprises:

training each teacher model using matched features extracted by the matched feature extractor from teacher training data for the corresponding one of the plurality of the domains.

4. The method of claim 2, wherein the unified feature extractor of the student model extracts an unified feature from an input signal in any one of the plurality of the domains by unifying physical meanings of features between the plurality of the domains.

5. The method of claim 4, wherein the unified feature extractor of the student model includes a hybrid normalization parameter set used in common for the plurality of the domains.

6. The method of claim 4, wherein a first unified feature extracted for a first domain has a plurality of elements and a second unified feature extracted for a second domain has a part of elements corresponding to a part of the elements for the first domain and a remaining part of elements corresponding to a remaining part of the elements for the first domain and having a predetermined value.

7. The method of claim 2, wherein the training of the student model comprises:

extracting an unified feature by the unified feature extractor from training data in each of the plurality of training data collections; and

using the unified feature and a soft label associated with the unified feature as an input to the student model and privileged information, respectively.

8. The method of claim 2, wherein the plurality of the teacher models and the student model are acoustic models and the plurality of the domains has difference in sampling condition of an input speech signal.

9. The method of claim 2, wherein the plurality of the teacher models and the student model are image processing models and the plurality of the domains has difference in color mode of an input image signal.

10. The method of claim 1, wherein the student model is a neural network based model.

11. A computer system for constructing a model supporting a plurality of domains, the computer system comprising:

a memory storing program instructions;

a processing circuitry in communications with the memory for executing the program instructions, wherein the program instructions are configured to;

prepare a plurality of teacher models, wherein each teacher model is specialized for different one of the plurality of the domains;

obtain a plurality of training data collections, wherein each of the plurality of training data collections is collected for a different one of the plurality of the domains;

input training data from each of the plurality of training data collections into a corresponding one of the plurality of the teacher models to generate a plurality of soft label sets; and

train a student model using the plurality of the soft label sets.

12. The computer system of claim 11, wherein each teacher model is connected to a matched feature extractor for corresponding one of the plurality of the domains and the student model is connected to a unified feature extractor, the unified feature extractor being common at least partially to the plurality of the domains.

13. The computer system of claim 12, wherein the matched feature extractor of each teacher model extracts a matched feature from an input signal in the corresponding one of the plurality of the domains and the processing circuitry is further configured to:

train each teacher model using matched features extracted by the matched feature extractor from teacher training data for the corresponding one of the plurality of the domains to prepare the plurality of the teacher models.

14. The computer system of claim 12, wherein the unified feature extractor of the student model extracts an unified feature from an input signal in any one of the plurality of the domains by unifying physical meanings of features between the plurality of the domains.

15. The computer system of claim 14, wherein the unified feature extractor of the student model includes a hybrid normalization parameter set used in common for the plurality of the domains.

16. The computer system of claim 12, wherein the processing circuitry is further configured to:

extract an unified feature by the unified feature extractor from training data in each of the plurality of training data collections; and

use the unified feature and a soft label associated with the unified feature as an input to the student model and privileged information, respectively.

17. The computer system of claim 12, wherein the plurality of the teacher models and the student model are acoustic models and the plurality of the domains has difference in sampling condition of an input speech signal.

18. A computer program product constructing a model supporting a plurality of domains, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:

preparing a plurality of teacher models, each teacher model being specialized for different one of the plurality of the domains;

obtaining a plurality of training data collections, each of the plurality of training data collections being collected for a different one of the plurality of the domains;

inputting training data from each of the plurality of the training data collections into a corresponding one of the plurality of the teacher models to generate a plurality of soft label sets; and

training a student model using the plurality of the soft label sets.

19. The computer program product of claim 18, wherein each teacher model is connected to a matched feature extractor for corresponding one of the plurality of the domains and the student model is connected to a unified feature extractor, the unified feature extractor being common at least partially to the plurality of the domains.

20. The computer program product of claim 19, wherein the matched feature extractor of each teacher model extracts a matched feature from an input signal in the corresponding one of the plurality of the domains and the preparing of the plurality of the teacher models comprises:

training each teacher model using matched features extracted by the matched feature extractor from teacher training data for the corresponding one of the plurality of the domains.

21. The computer program product of claim 19, wherein the unified feature extractor of the student model extracts an unified feature from an input signal in any one of the plurality of the domains by unifying physical meanings of features between the plurality of the domains.

22. The computer program product of claim 21, wherein the unified feature extractor of the student model includes a hybrid normalization parameter set used in common for the plurality of the domains.

23. The computer program product of claim 19, wherein the training of the student model comprises:

extracting an unified feature by the unified feature extractor from training data in each of the plurality of training data collections; and

using the unified feature and a soft label associated with the unified feature as an input to the student model and privileged information, respectively.

24. The computer program product of claim 19, wherein the plurality of the teacher models and the student model are acoustic models and the plurality of the domains has difference in sampling condition of an input speech signal.