Adapted vocabularies for matching image signatures with fisher vectors

• US 9,607,245 B2
• Filed: 12/02/2014
• Issued: 03/28/2017
• Est. Priority Date: 12/02/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Tariq, Usman, Rodriguez Serrano, Jos Antonio, Perronnin, Florent C.
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US14/557,891
• Publication Number:
  US 20160155020A1
• Time in Patent Office: 847 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names

Abstract

A method includes adapting the universal generative model of local descriptors to a first camera to obtain a first camera-dependent generative model. The same universal generative model is also adapted to a second camera to obtain a second camera-dependent generative model. From a first image captured by the first camera, a first image-level descriptor is extracted, using the first camera-dependent generative model. From a second image captured by the second camera, a second image-level descriptor is extracted using the second camera-dependent generative model. A similarity is computed between the first image-level descriptor and the second image-level descriptor. Information is output, based on the computed similarity. The adaptation allows differences between the image-level descriptors to be shifted towards deviations in image content, rather than the imaging conditions.

Claims

1. A method comprising:

providing a universal generative model of local descriptors;

adapting the universal generative model to a first camera to obtain a first camera-dependent generative model using local descriptors extracted from each of a set of training images captured by the first camera;

adapting the universal generative model to a second camera to obtain a second camera-dependent generative model using local descriptors extracted from each of a set of training images captured by the second camera or using the universal generative model as the second camera-dependent generative model;

from a first test image captured by the first camera, extracting a first image-level descriptor, the first image-level descriptor being a fixed-length vectorial representation of the first test image generated by aggregating local descriptors extracted from the first image into a fixed-length representation using the first camera-dependent generative model;

from a second test image captured by the second camera, extracting a second image-level descriptor, the second image-level descriptor being a fixed-length vectorial representation of the second test image generated by aggregating local descriptors extracted from the second image into a fixed-length representation using the second camera-dependent generative model;

computing a similarity between the first image-level descriptor and the second image-level descriptor; and

outputting information based on the computed similarity, wherein at least one of the adapting the universal generative model to the first and second cameras, extracting the first and second image-level descriptors and the computing of the similarity is performed with a computer processor.

2. The method of claim 1, wherein the universal generative model is a Gaussian Mixture Model.

3. The method of claim 2, wherein the first and second camera-dependent generative models are Gaussian Mixture Models, each comprising a same number of Gaussian functions as the universal generative model.

4. The method of claim 1, wherein the universal generative model includes a set of parameters and the adaptation comprises adapting the parameters of the universal generative model to generate parameters of the first and second camera-dependent generative models.

5. The method of claim 4, wherein the parameters are selected from the group consisting of weight, mean vector, and covariance matrix of each Gaussian of the respective generative model.

6. The method of claim 5, wherein the first image-level descriptor comprises gradients with respect to at least one of the parameters of the first camera-dependent generative model and the second image-level descriptor comprises gradients with respect to at least one of the parameters of the second camera-dependent generative model.

7. The method of claim 1, wherein the first and second image-level descriptors are Fisher vectors.

8. The method of claim 1, wherein the computing the similarity comprises computing a cosine distance between the first and second image-level descriptors.

9. The method of claim 1, wherein a reidentification of an object in the first and second images is confirmed based on the computed similarity meeting a threshold similarity.

10. The method of claim 1, wherein when the similarity meets a threshold similarity, the method comprises extracting license plate information from at least one of the first and second images, the output information comprising the extracted license plate information.

11. The method of claim 1, wherein the adapting of the universal generative model to the first and second cameras is performed with an unsupervised adaptation method.

12. The method of claim 11, wherein the adapting of the universal generative model to the first and second cameras comprises maximum a posteriori (MAP) adaptation of parameters of the universal generative model.

13. The method of claim 1, wherein the adapting of the universal generative model to the first and second cameras comprises extracting local descriptors from images captured by the first and second cameras, the local descriptors from the images captured by the first camera being used to adapt the universal generative model to the first camera, the local descriptors from the images captured by the second camera being used to adapt the universal generative model to the second camera.

14. The method of claim 1, wherein the extracting of the first and second image-level descriptor comprises extracting patches from each of the first and second images and, for each patch, generating a local descriptor, each image-level descriptor estimating a deviation of the respective local descriptors from the respective camera-dependent generative model.

15. The method of claim 1, wherein the providing of the universal generative model comprises training the universal generative model on local descriptors extracted from training images.

16. A computer program product comprising non-transitory memory storing instructions which, when executed by a computer, perform the method of claim 1.

17. A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

18. A system comprising:

memory which stores a universal generative model of local descriptors, the universal generative model being a Gaussian Mixture Model including parameters for each of the Gaussians, the parameters including a mixture weight, a mean vector, and a covariance matrix; and

an adaptation component which adapts the parameters of the universal generative model to a first camera to obtain a first camera-dependent generative model and adapts the parameters of the universal generative model to a second camera to obtain a second camera-dependent generative model; and

a processor which implements the adaptation component.

19. The system of claim 18, further comprising:

a matching component which computes a similarity between a first image-level descriptor and a second image-level descriptor, the first image-level descriptor having been extracted from a first image captured by the first camera, using the first camera-dependent generative model, the second image-level descriptor having been extracted from a second image captured by the second camera, using the second camera-dependent generative model.

20. The system of claim 18, further comprising:

a signature generation component which extracts at least one of a first image-level descriptor and a second-image-level descriptor, the first image-level descriptor being extracted from a first image captured by the first camera using the first camera-dependent generative model, the second image-level descriptor being extracted from a second image captured by the second camera using the second camera-dependent generative model.