Domain adaptation for image classification with class priors

• US 9,710,729 B2
• Filed: 09/04/2014
• Issued: 07/18/2017
• Est. Priority Date: 09/04/2014
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Chidlovskii, Boris, Csurka, Gabriela
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US14/477,215
• Publication Number:
  US 20160070986A1
• Time in Patent Office: 1,048 Days
• Field of Search: None
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06F 18/2155 characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

In camera-based object labeling, boost classifier ƒ^T(x)=Σ_r=1^Mβ_rh_r(x) is trained to classify an image represented by feature vector x using a target domain training set D_T of labeled feature vectors representing images acquired by the same camera and a plurality of source domain training sets D_S1, . . . , D_SN acquired by other cameras. The training applies an adaptive boosting (AdaBoost) algorithm to generate base classifiers h_r(x) and weights β_r. The r^th iteration of the AdaBoost algorithm trains candidate base classifiers h_r^k(x) each trained on a training set D_T∪D_Sk, and selects h_r(x) from previously trained candidate base classifiers. The target domain training set D_T may be expanded based on a prior estimate of the labels distribution for the target domain. The object labeling system may be a vehicle identification system, a machine vision article inspection system, or so forth.

Claims

1. A labeling system comprising:

an electronic data processing device configured to label an image to be labeled belonging to a target domain by operations including;

training a boost classifier f^T(x)=Σ

_r=1^Mβ

_rh_r(x) to classify an image belonging to the target domain and represented by a feature vector x in a feature space X, the training using a target domain training set D_T comprising labeled feature vectors in the feature space X representing images belonging to the target domain and a plurality of source domain training sets D_S1, . . . , D_SN where N≧

2 comprising labeled feature vectors in the feature space X representing images belonging to source domains S₁, . . . , S_N respectively, the training comprising expanding the target domain training set D_T based on a prior estimate of the labels distribution for the target domain and, after expanding the target domain training set D_T, applying an adaptive boosting (AdaBoost) algorithm to generate the base classifiers h_r(x) and the base classifier weights β

_r of the boost classifier f^T(x), wherein the r^th iteration of the AdaBoost algorithm includes (i) performing N sub-iterations in which the k^th sub-iteration trains a candidate base classifier h_r^k(x) on a training set combining the target domain training set D_T and the source domain training set D_Sk and (ii) selecting h_r(x) as the candidate base classifier with lowest error for the target domain training set D_T;

computing a feature vector x_in in the feature space X representing the image to be labeled; and

generating a label for the image to be labeled by operations including evaluating f^T(x_in)=Σ

_r=1^Mβ

_rh_r(x_in).

2. The labeling system of claim 1 wherein the labeling system is a camera-based object labeling system further comprising:

a system camera arranged to acquire images of objects;

wherein the target domain is defined as the domain of images of objects acquired by the system camera and the image to be labeled is an image of an object to be labeled acquired by the system camera;

wherein each source domain S₁, . . . , S_N is defined as the domain of images of objects acquired by a camera other than the system camera; and

wherein the electronic data processing device is further configured to generate a label for the object to be labeled based on the label generated for the image to be labeled.

3. The camera-based object labeling system of claim 2 further comprising:

a display device operatively connected with the electronic data processing device to display the image of the object to be labeled together with the label generated for the object.

4. The labeling system of claim 1 wherein the k^th sub-iteration trains the candidate base classifier h_r^k(x) on a union D_T∪

D_Sk of the target domain training set D_T and the source training set D_Sk.

5. The labeling system of claim 1 wherein the training of the boost classifier f^T(x)=Σ

_r=1^Mβ

_rh_r(x) further comprises;

before applying the Adaboost algorithm, performing unsupervised source-target domain alignment to align the target domain training set D_T and the source training sets D_Sk, k=1, . . . , N.

6. The labeling system of claim 1 wherein the r^th iteration of the AdaBoost algorithm further includes:

(iii) updating weight vectors w_i^sk for the training instances i of the source training sets D_Sk,k=1, . . . , N based on the error for the target domain training set D_T of the candidate base classifier selected as the base classifier h_r(x).

7. The labeling system of claim 1 wherein the Adaboost algorithm maintains a queue PQ of candidate base classifiers across iterations of the Adaboost algorithm, and the selecting operation (ii) includes:

(ii)(a) selecting h_r(x) as the candidate base classifier in the queue PQ with lowest error for the target domain training set D_T; and

(ii)(b) removing the selected candidate base classifier from the queue PQ.

8. The labeling system of claim 1 wherein the expanding of the target domain training set D_T comprises:

adding to the target domain training set D_T additional synthesized instances with different labels wherein the synthesized instances have initialized weight vectors for the Adaboost algorithm computed based on label probabilities generated using the prior estimate of the labels distribution.

9. A labeling method for labeling an image to be labeled belonging to a target domain, the image labeling method comprising:

computing feature vectors representing target domain training images belonging to the target domain;

labeling the target domain training images using labels selected from a set of labels to generate a target domain training set D_T comprising labeled feature vectors representing the target domain training images;

receiving a plurality of source domain training sets D_S1, . . . , D_SN where N≧

1 comprising feature vectors representing images belonging to source domains different from the target domain that are labeled using labels selected from the set of labels;

performing unsupervised source-target domain alignment to align the target domain training set D_T and the source training sets D_Sk, k=1, . . . , N;

training a boost classifier f^T(x)=Σ

_r=1^Mβ

_rh_r(x) to classify an image belonging to the target domain and represented by a feature vector x, the training using the aligned target domain training set D_T and plurality of source domain training sets D_S1, . . . , D_SN, the training comprising expanding the target domain training set D_T based on a prior estimate of the labels distribution for the target domain and, after expanding the target domain training set D_T, applying an adaptive boosting (AdaBoost) algorithm to generate the base classifiers h_r(x) and the base classifier weights β

_r of the boost classifier f^T(x), where r=1, . . . , M; and

computing a feature vector x_in representing the image to be labeled; and

generating a label for the image to be labeled by operations including evaluating f^T(x_in)_r=1^Mβ

_rh_r(x_in);

wherein the feature vector computing operations, the training operation, and the generating operation are performed by an electronic data processing device.

10. The labeling method of claim 9 wherein N≧

2 and the r^th iteration of the AdaBoost algorithm includes (i) training candidate base classifiers h_r^k(x) wherein h_r^k(x) is trained on a training set combining the target domain training set D_T and the source training set D_Sk and (ii) selecting h_r(x) from a pool of trained candidate base classifiers based on a target domain error metric.

11. The labeling method of claim 10 wherein the operation (ii) selects h_r(x) from one of:

the pool of trained candidate base classifiers h_r^k(x) trained during the r^th iteration; and

the pool of trained candidate base classifiers trained during the r^th iteration and any earlier iterations, wherein the operation (ii) further includes removing the candidate base classifier selected as h_r(x) from the pool.

12. The labeling method of claim 10 wherein the operation (i) includes performing k=1, . . . , N sub-iterations in which the k^th sub-iteration trains a candidate base classifier h_r^k(x) on a training set combining the target domain training set D_T and the source training set D_Sk.

13. The labeling method of claim 9 wherein the set of labels is a set of object labels, and the image to be labeled, the target domain training images, and the images belonging to source domains different from the target domainare images of objects, and the labeling method further comprises:

displaying, on a display device, the image to be labeled with the generated object label.

14. The labeling method of claim 9 wherein the set of labels is a set of object labels, and the image to be labeled, the target domain training images, and the images belonging to source domains different from the target domainare images of objects, and the labeling method further comprises:

generating and displaying, on a display device, an object inspection result based on the generated label.

15. The labeling method of claim 13 wherein one of:

the objects are vehicles and the vehicle inspection result is a number of occupants of the vehicle;

or

the objects are articles of manufacture being processed by an assembly line and the article inspection result is a pass or fail result.

16. The labeling method of claim 9 wherein the set of labels is a set of vehicle labels, and the image to be labeled, the target domain training images, and the images belonging to source domains different from the target domainare images of vehicles, and the labeling method further comprises:

assigning a toll or fee associated with the generated vehicle label wherein the assigning is performed by the electronic data processing device; and

displaying, on a display device operatively connected with the electronic data processing device, the image to be labeled together with the assigned toll or fee.

17. The labeling method of claim 9 wherein the r^th iteration of the AdaBoost algorithm further includes:

updating weight vectors w_i^Sk for the training instances i of the source training sets D_Sk, k=1, . . . , N based on the error for the target domain training set D_T of the base classifier h_r(x).

18. The labeling method of claim 9 wherein the expanding of the target domain training set D_T comprises:

adding to the target domain training set D_T additional synthesized instances with different labels wherein the synthesized instances have initialized weight vectors for the Adaboost algorithm computed based on label probabilities generated using the prior estimate of the labels distribution.

19. A non-transitory storage medium storing instructions executable by an electronic data processing device to perform a camera-based object labeling method to label an object based on an image of the object acquired using a target camera, the camera-based object labeling method including the operations of:

training a boost classifier f^T(x)=Σ

_r=1^Mβ

_rh_r(x) to classify an image acquired by the target camera and represented by a feature vector x, the training using a target domain training set D_T comprising labeled feature vectors representing images acquired by the target camera and a plurality of source domain training sets D_S1, . . . , D_SN where N≧

2 comprising labeled feature vectors representing images of objects acquired by cameras other than the target camera, the training comprising expanding the target domain training set D_T based on a prior estimate of the labels distribution for the target domain and, after the expanding, applying an adaptive boosting (AdaBoost) algorithm to generate the base classifiers h_r(x) and the base classifier weights β

_r of the boost classifier f^T(X), wherein the AdaBoost algorithm includes r=1, . . . , M iterations and the r^th iteration includes training a plurality of candidate base classifiers h_r^k(x) wherein each candidate base classifier h_r^k(x) is trained on a training set D_T∪

D_Sk and selecting h_r(x) from a set of previously trained candidate base classifiers; and

computing a feature vector x_in representing the image of the object; and

generating a label for the object by evaluating f^T(x_in)=Σ

_r=1^Mβ

_rh_r(x_in).

20. The non-transitory storage medium of claim 19 wherein the expanding of the target domain training set D_T comprises:

adding to the target domain training set D_T additional synthesized instances with different labels wherein the synthesized instances have initialized weight vectors for the Adaboost algorithm computed based on label probabilities generated using the prior estimate of the labels distribution.

21. The non-transitory storage medium of claim 19 wherein the r^th iteration of the AdaBoost algorithm further includes:

(iii) updating weight vectors w_i^Sk for the training instances i of the source training sets D_Sk,k=1, . . . , N based on the error for the target domain training set D_T of the candidate base classifier selected as the base classifier h_r(x).