Transductive adaptation of classifiers without source data

• US 10,354,199 B2
• Filed: 12/07/2015
• Issued: 07/16/2019
• Est. Priority Date: 12/07/2015
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Clinchant, Stephane, Csurka, Gabriela, Chidlovskii, Boris
• Primary Examiner(s): Afshar, Kamran
• Assistant Examiner(s): Hoover, Brent Johnston
• Application Number:
  US14/960,869
• Publication Number:
  US 20170161633A1
• Time in Patent Office: 1,317 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

A classification method includes receiving a collection of samples, each sample comprising a multidimensional feature representation. A class label prediction for each sample in the collection is generated with one or more pretrained classifiers. For at least one iteration, each multidimensional feature representation is augmented with a respective class label prediction to form an augmented representation, a set of corrupted samples is generated from the augmented representations, and a transformation that minimizes a reconstruction error for the set of corrupted samples is learned. An adapted class label prediction for at least one of the samples in the collection is generated using the learned transformation and information is output, based on the adapted class label prediction. The method is useful in predicting labels for target samples where there is no access to source domain samples that are used to train the classifier and no access to target domain training data.

Claims

1. A classification method comprising:

providing access to a pretrained classifier which has been trained on source samples in a source domain and their respective class labels,

thereafter, receiving a collection of unlabeled target samples for a target domain, different from the source domain, each target sample comprising a multidimensional feature representation;

with the pretrained classifier, generating a class label prediction for each of the target samples in the collection;

for at least one iteration, without access to source samples from the source domain,

augmenting each target sample multidimensional feature representation with a respective class label prediction output by the pretrained classifier to form an augmented representation,

generating a set of corrupted target samples from the augmented representations, and

learning a transformation that minimizes a reconstruction error for the set of corrupted target samples;

generating an adapted class label prediction for at least one of the target samples in the collection using the learned transformation; and

outputting information based on the adapted class label prediction,

wherein the augmenting, generating a set of corrupted samples, learning a transformation, and generating an adapted class label prediction are performed with a hardware processor.

2. The method of claim 1, wherein the generation of the corrupted samples includes removing features from the augmented representations with a predefined probability.

3. The method of claim 1, wherein the adapted class label prediction is generated without access to data that was used to train the classifier.

4. The method of claim 1, wherein the method is performed without access to labeled samples in the target domain.

5. The method of claim 1, wherein at least one iteration includes a plurality of iterations and wherein in a later one of the iterations, the augmented representation is based on the adapted class label predictions of a previous one of the iterations.

6. The method of claim 5, further comprising applying a non-linear function to the class predictions between the previous and later iterations.

7. The method of claim 1, wherein the learning of the transformation includes reconstructing the augmented representations from the corrupted samples and learning a transformation which minimizes an error between the corrupted samples and reconstructed augmented representations.

8. The method of claim 1 wherein the learning of the transformation includes learning W=min_W∥

Z′

−

W{tilde over (Z)}∥

², where W represents the transformation, Z′

represents a set of augmented representations reconstructed from the corrupted samples, and {tilde over (Z)} represents the set of corrupted samples.

9. The method of claim 1, wherein the generating of the adapted class label prediction for the at least one sample in the collection includes computing a product of the transformation and the multidimensional feature representation.

10. The method of claim 1, wherein the transformation comprises a matrix.

11. The method of claim 1, wherein the generating of the set of corrupted samples from the augmented representations comprises generating a greater number of corrupted samples than the number of samples in the collection of samples.

12. The method of claim 1, wherein the output information comprises at least one of a label for the sample and a ranking of the sample.

13. The method of claim 1, further comprising implementing a process based on the adapted class label prediction and the outputting of information includes outputting information generated in the process.

14. The method of claim 1, wherein the samples each comprise a representation of at least one of an image and a text sequence.

15. A computer program product comprising a non-transitory recording medium storing instructions which, when executed by a processor, perform the method of claim 1.

16. A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

17. A classification system comprising:

a first prediction component which uses a pretrained classifier to generate a class label prediction for each sample in a collection of unlabeled samples, each sample comprising a multidimensional feature representation;

a learning component which learns a transformation, the learning component including a stack of autoencoders, each of the autoencoders including an encoder which corrupts input feature vectors and a decoder which reconstructs the input feature vectors from the corrupted feature vectors, the transformation being learned to minimize the reconstruction error, wherein in a first of the layers, the input feature vectors include the multidimensional feature representations augmented by their class label predictions output by the pretrained classifier and in a second of the layers, the input feature vectors are based on class label predictions output by the first layer;

a second prediction component which generates an adapted class label prediction for at least one of the samples in the collection using the learned transformation;

an output component which outputs information based on the adapted class label prediction; and

a hardware processor which implements the components.

18. The classification system of claim 17 wherein the system includes the pretrained classifier but does not have access to training samples which were used to learn the classifier.

19. A classification method comprising:

receiving a collection of target samples in a target domain, each sample comprising a multidimensional feature representation;

with a pretrained classifier trained on labeled source samples in a source domain, generating a class label prediction for each of the target samples in the collection;

with a hardware processor, in a first of a plurality of iterations,

augmenting each multidimensional feature representation with a respective one of the class label predictions generated by the pretrained classifier to form an augmented representation,

generating a set of corrupted samples from the augmented representations,

learning a transformation that minimizes a reconstruction error for the set of corrupted samples, and

generating an adapted class label prediction for each of the target samples in the collection using the learned transformation;

in at least a second of the plurality of iterations, repeating the generating of a set of corrupted samples, learning a transformation, and generating adapted class label predictions, wherein the set of corrupted samples are generated from augmented representations that are based on adapted class label predictions from a preceding iteration; and

outputting information based on the adapted class label predictions of one of the plurality of iterations.

20. The classification method of claim 19, wherein the method is performed without access to labeled samples in the source domain and target domain.