Image segmentation based on approximation of segmentation similarity

• US 8,917,910 B2
• Filed: 01/16/2012
• Issued: 12/23/2014
• Est. Priority Date: 01/16/2012
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Rodriguez Serrano, Jose Antonio
• Primary Examiner(s): FUJITA, KATRINA R
• Application Number:
  US13/351,038
• Publication Number:
  US 20130182909A1
• Time in Patent Office: 1,072 Days
• Field of Search: 382/105, 382/159, 382/218, 382/305
• US Class Current: 382/105
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/148 Segmentation of character regions
• CPC Class Codes: G06V 30/19127 Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods

Abstract

A system and a method for image segmentation use segmentation maps of one or more similar images as a basis for the segmentation. The method includes generating an image signature for an input image to be segmented and identifying at least one similar image from a set of images, based on the image signature of the input image and image signatures of images in the set of images. The similarity may be computed after first projecting the image signatures into a feature space where similarity is more likely to agree with segmentation map similarity. The input image is segmented, based on the segmentation map of one or more of the at least one identified similar images.

Claims

1. A method for image segmentation comprising:

learning a projection matrix using segmentation maps and image signatures of a set of training images;

generating an image signature for an input image to be segmented;

projecting the image signature of the image to be segmented into a feature space with the learned projection matrix;

identifying at least one similar image from a set of images, based on the projected image signature of the input image and projected image signatures of images in the set of images projected with the learned projection matrix, the identifying comprising computing a similarity between the projected image signature and the projected image signatures of the images in the set, wherein the set of images comprises a set of license plate images and respective segmentation maps, each segmentation map including a foreground region, which corresponds to an object of interest, and a background region, which corresponds to the rest of the image;

segmenting the input image based on the segmentation map of at least one of the at least one similar images, the segmentation including identifying a foreground region of the input image; and

identifying a license plate number from the foreground region of the segmented input image.

2. A method for image segmentation comprising:

providing a learned projection matrix learned using segmentation maps and image signatures of a set of training images, the learning of the projection matrix including, for a plurality of iterations;

drawing a triplet of entries from the set of training images, each entry comprising a segmentation map and an image signature for an image in the training set;

computing a similarity between a first pair of the entries, based on their segmentation maps;

computing a similarity between a second pair of the entries, based on their segmentation maps;

generating a first ranking for the first and second pairs based on the computed segmentation map similarities;

computing a similarity between the first pair of the entries, based on their image signatures projected with a current projection matrix;

computing a similarity between a second pair of the entries, based on their image signatures projected with the current projection matrix;

generating a second ranking for the first and second pairs based on the computed projected image signature similarities; and

updating the current projection matrix if the first and second rankings do not agree; and

after the plurality of iterations, outputting the updated projection matrix as the projection matrix;

generating an image signature for an input image to be segmented;

projecting the image signature of the image signature to be segmented into a feature space with the learned projection matrix;

identifying at least one similar image from the set of images, based on the image signature of the input image and image signatures of images in the set of images, comprising computing a similarity between the projected image signature and projected image signatures of the images in the set of images, projected with the learned projection matrix; and

segmenting the input image based on a segmentation map of at least one of the at least one similar images.

3. The method of claim 2, wherein at least one of the generating, identifying and segmenting is performed with a computer processor.

4. The method of claim 2, wherein the generating of the image signature comprises generating a multidimensional vector based on features extracted from patches of the image.

5. The method of claim 2, wherein the identifying comprises identifying the top K most similar images from the set of images, where K is greater than 1.

6. The method of claim 2, further comprising establishing a threshold similarity for segmentation maps and discarding the triplet if one of the first and second pairs does not exceed the threshold.

7. The method of claim 2, wherein the segmenting of the image comprises generating a segmentation map comprising a first region of the image and a second region of the image.

8. The method of claim 2, further comprising providing a threshold similarity for the at least one similar image and, if the threshold is not met by any of the images in the set of images, identifying no similar images.

9. The method of claim 2, wherein the generating of the image signature for the input image comprise generating a single global image signature for the input image and wherein the identifying of the at least one similar image from the set of images is based on the single global image signature.

10. The method of claim 2, wherein the set of images comprises license plate images.

11. The method of claim 10, further comprising identifying a license plate number from a foreground region of the segmented input image.

12. The method of claim 2, wherein the set of images comprises synthesized license plate images.

13. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, causes the computer to perform the method of claim 2.

14. A system for image segmentation comprising memory which stores instructions for performing the method of claim 2 and a processor in communication with the memory for executing the instructions.

15. A system for image segmentation comprising:

a signature generator configured for generating an image signature for an input image to be segmented;

memory which stores a learned projection matrix for projecting image signatures into a feature space in which projected signature similarity is more likely to correlate with a similarity of segmentation maps of images, the learned projection matrix having been estimated using image signatures and segmentation maps of a set of segmented training images, each segmentation map including a foreground region, which corresponds to an object of interest, wherein the learned projection matrix is learned using segmentation maps and image signatures of a set of training images, the learning of the projection matrix including, for a plurality of iterations;

drawing a triplet of entries from the set of training images, each entry comprising a segmentation map and an image signature for an image in the training set;

computing a similarity between a first pair of the entries, based on their segmentation maps;

computing a similarity between a second pair of the entries, based on their segmentation maps;

generating a first ranking for the first and second pairs based on the computed segmentation map similarities;

computing a similarity between the first pair of the entries, based on their image signatures projected with a current projection matrix;

computing a similarity between a second pair of the entries, based on their image signatures projected with the current projection matrix;

generating a second ranking for the first and second pairs based on the computed projected image signature similarities; and

updating the current projection matrix if the first and second rankings do not agree; and

after the plurality of iterations, outputting the updated projection matrix as the projection matrix;

a retrieval component configured for identifying at least one similar image from a set of images, based on the image signature of the input image and image signatures of images in the set of images, the signatures being projected with the projection matrix; and

a segmentation component configured for segmenting the input image based on a segmentation map of at least one of the at least one similar images; and

;

a processor for implementing the signature generator, retrieval component, and segmentation component.