IMAGE SEGMENTATION BASED ON APPROXIMATION OF SEGMENTATION SIMILARITY

• US 20130182909A1
• Filed: 01/16/2012
• Published: 07/18/2013
• Est. Priority Date: 01/16/2012
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Rodriguez-Serrano, Jos Antonio
• Granted Patent: US 8,917,910 B2
• Time in Patent Office: Days
• US Class Current: 382/105
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/148 Segmentation of character regions
• CPC Class Codes: G06V 30/19127 Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods

Abstract

A system and a method for image segmentation use segmentation maps of one or more similar images as a basis for the segmentation. The method includes generating an image signature for an input image to be segmented and identifying at least one similar image from a set of images, based on the image signature of the input image and image signatures of images in the set of images. The similarity may be computed after first projecting the image signatures into a feature space where similarity is more likely to agree with segmentation map similarity. The input image is segmented, based on the segmentation map of one or more of the at least one identified similar images.

Claims

1. A method for image segmentation comprising:

generating an image signature for an input image to be segmented;

identifying at least one similar image from a set of images, based on the image signature of the input image and image signatures of images in the set of images; and

segmenting the input image based on a segmentation map of at least one of the at least one similar images.

2. The method of claim 1, wherein at least one of the generating, identifying and segmenting is performed with a computer processor.

3. The method of claim 1, wherein the generating of the image signature comprises generating a multidimensional vector based on features extracted from patches of the image.

4. The method of claim 1, wherein the identifying comprises identifying the top K most similar images from the set of images, where K is greater than 1.

5. The method of claim 1, further comprising projecting the image signature of the image signature to be segmented into a feature space with a learned projection matrix and wherein the identifying comprises computing a similarity between the projected image signature and projected image signatures of the images in the set projected with the learned projection matrix.

6. The method of claim 5, further comprising learning the projection matrix using segmentation maps and image signatures of a set of training images.

7. The method of claim 6, wherein the learning of the projection matrix comprises:

for a plurality of iterations;

drawing a triplet of entries set of training images, each entry comprising a segmentation map and an image signature for an image in the training set;

computing a similarity between a first pair of the entries, based on their segmentation maps;

computing a similarity between a second pair of the entries, based on their segmentation maps;

generating a first ranking for the first and second pairs based on the computed segmentation map similarities;

computing a similarity between the first pair of the entries, based on their image signatures projected with a current projection matrix;

computing a similarity between a second pair of the entries, based on their image signatures projected with the current projection matrix;

generating a second ranking for the first and second pairs based on the computed projected image signature similarities; and

updating the current projection matrix if the first and second rankings do not agree; and

after the plurality of iterations, outputting the updated projection matrix as the projection matrix.

8. The method of claim 7, further comprising establishing a threshold similarity for segmentation maps and discarding the triplet if one of the first and second pairs does not exceed the threshold.

9. The method of claim 1, wherein the identifying of the at least one similar image from a set of image is based on similarities between a projected image signature of the input image and projected image signatures of the images in the set of images computed in a feature space in which similarities between projected image signatures are more likely to correlate with similarities based on segmentation maps than are computed similarities between image signatures.

10. The method of claim 1, wherein the segmenting of the image comprises generating a segmentation map comprising a first region of the image and a second region of the image.

11. The method of claim 1, further comprising providing a threshold similarity for the at least one similar image and, if the threshold is not met by any of the images in the set of images, identifying no similar images.

12. The method of claim 1, wherein the generating of the image signature for the input image comprise generating a single global image signature for the input image and wherein the identifying of the at least one similar image from the set of images is based on the single global image signature.

15. The method of claim 1, wherein the set of ages comprises synthesized images.

16. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, causes the computer to perform the method of claim 1.

17. A system for image segmentation comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

13. The method of aim 1, wherein the set of ages comprises license plate images.

14. The method of claim 13, further comprising identifying a license plate number from a foreground region of the segmented input image.

18. A system for image segmentation comprising:

a signature generator configured for generating an image signature for an input image to be segmented;

memory which stores a learned projection matrix for projecting image signatures into a feature space in which projected signature similarity is more likely to correlate with a similarity of segmentation maps of images;

a retrieval component configured for identifying at least one similar image from a set of images, based on the image signature of the input image and image signatures of images in the set of images, the signatures being projected with the projection matrix; and

a segmentation component configured for segmenting the input image based on a segmentation map of at least one of the at least one similar images.

19. The system of claim 18, further comprising a processor for implementing the signature generator, retrieval component, and segmentation component.

20. A method for forming a system for image segmentation comprising:

providing a signature generator for generating an image signature for an input image to be segmented;

learning a projection matrix for projecting image signatures into a feature space in which a ranked similarity of pairs of projected image signatures of images in a training set is more likely to agree with ranked similarity of segmentation maps of the pairs of images than a ranked similarity of pairs of image signatures without projection;

storing projected image signatures for each of set of images, the projected image signatures being generated with the learned projection matrix;

providing a retrieval component for identifying at least one similar image from the set of images, similarity being computed based on the projected image signature of the input image and the projected image signatures of the images in the set of images, the projected image signature of the input image being generated by projecting the input image'"'"'s image signature with the projection matrix; and

providing a segmentation component for segmenting the input image based on a segmentation map of at least one of the at least one similar images.

21. A method for image segmentation in which an output segmentation map for an input image is equal to a segmentation map of a most similar one of a set of images or where the output segmentation map is derived from the segmentation maps of the best ranked K similar ones of the set of images, similarity being based on a projected image signature of the input image and projected image signatures of the set of images using a learned projection matrix.