System and method for image color transfer based on target concepts

• US 8,553,045 B2
• Filed: 09/24/2010
• Issued: 10/08/2013
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Skaff, Sandra, Murray, Naila, Marchesotti, Luca, Perronnin, Florent
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US12/890,049
• Publication Number:
  US 20120075329A1
• Time in Patent Office: 1,110 Days
• Field of Search: 345/593, 345/589
• US Class Current: 345/593
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G09G 2320/0606 Manual adjustment
• CPC Class Codes: G09G 2320/0666 for control of colour parameters, e.g. colour temperature
• CPC Class Codes: G09G 5/06 using colour palettes, e.g. look-up tables

Abstract

A method for color transfer includes retrieving a concept color palette from computer memory corresponding to a concept selected by a user. The concept color palette includes a first set of colors, which may be statistically representative of colors of a set of predefined color palettes which have been associated with the concept. The method further includes computing an image color palette for an input image. The image color palette includes a second set of colors that are representative of pixels of the input image. Colors of the image color palette are mapped to colors of the concept color palette to identify, for colors of the image color palette, a corresponding color in the concept color palette. A transformation is computed based on the mapping. For pixels of the input image, modified color values are computed, based on the computed transformation, to generate a modified image.

Claims

1. A method for color transfer comprising:

receiving a concept selected by a user from a predefined set of concepts;

retrieving a concept color palette from computer memory corresponding to the user-selected concept, the concept color palette comprising a first set of colors, each of a set of the concept color palettes having been automatically generated from a respective set of predefined color palettes;

computing an image color palette for an input image, the image color palette comprising a second set of colors that are representative of pixels of the input image;

mapping colors of the image color palette to colors of the concept color palette to identify, for colors of the image color palette, a corresponding color in the concept color palette;

computing a transformation based on the mapping;

for pixels of the input image, computing modified color values based on the computed transformation, to generate a modified image; and

outputting the modified image.

2. The method of claim 1, wherein the computing of the image color palette comprises computing a Gaussian mixture model (GMM) wherein means of Gaussian functions in the GMM comprise the second set of colors.

3. The method of claim 2, wherein the GMM is learned through Maximum Likelihood Estimation (MLE).

4. The method of claim 3, wherein observations for learning the GMM comprise color values for pixels in the image.

5. The method of claim 2, wherein the GMM is initialized with the first set of colors and corresponding weights.

6. The method of claim 1 wherein the computing of the image color palette comprises adaptation of the concept color palette to colors of the pixels of the image which provides the mapping of the colors of the image color palette to colors of the concept color palette.

7. The method of claim 1, wherein the concept color palette colors are learned from a set of predefined color palettes associated with the concept.

8. The method of claim 1, further comprising, for each of a set of concepts, learning a concept color palette based on colors of the set of predefined color palettes associated with the concept, each predefined color palette comprising a set of colors.

9. The method of claim 8, wherein the learning of the concept color palette comprises learning a GMM, wherein colors of the concept color palette are means of the GMM.

10. The method of claim 9, wherein observations for learning the GMM for the concept color palette are colors of the predefined color palettes.

11. The method of claim 1, wherein each of a set of concept color palettes is statistically representative of colors in a set of predefined color palettes and is associated with a respective description and wherein the retrieving of the concept color palette includes retrieving a concept color palette from the set of concept color palettes which has a description which matches a description selected by a user.

12. The method of claim 1, wherein the concept color palette includes at least eight colors.

13. The method of claim 1, wherein the image color palette includes at least eight colors.

14. The method of claim 1, wherein the image color palette includes at least as many colors as the concept color palette.

15. The method of claim 1, wherein the method includes providing for displaying a set of concept descriptions on a graphical user interface whereby a concept description is selectable by a user.

16. The method of claim 1, further comprising providing for displaying at least one of the image color palette, concept color palette, and modified image.

17. The method of claim 1, wherein the computing of the transformation comprises computing an affine transformation between the image color palette and the concept color palette.

18. The method of claim 1, further comprising providing for a user to select a degree of transfer whereby the transformation is adjustable.

19. A computer program product comprising a non-transitory recording medium storing instructions, which when executed by a computer, perform the method of claim 1.

20. An apparatus for performing the method of claim 1 comprising memory which stores instructions for performing the method and which stores a set of concept color palettes, and a processor in communication with the memory for executing the instructions.

21. The method of claim 1, wherein each concept color palette comprises a same number of colors.

22. A method for color transfer comprising:

receiving a concept selected by a user from a predefined set of concepts;

retrieving a concept color palette from computer memory corresponding to the user-selected concept, the concept color palette comprising a first set of colors;

computing an image color palette for an input image, the image color palette comprising a second set of colors that are representative of pixels of the input image;

mapping colors of the image color palette to colors of the concept color palette to identify, for colors of the image color palette, a corresponding color in the concept color palette, the mapping including identifying a color of the concept color palette for each color of the image color palette, whereby each image color palette color is associated with a single concept color palette color;

computing a transformation based on the mapping including computing a linear transformation between the image color palette and the mapped concept color palette;

for pixels of the input image, computing modified color values by applying the computed transformation to pixels of the image to generate a modified image; and

outputting the modified image.

23. An image adjustment apparatus comprising:

memory which stores a set of concept color palettes, each concept color palette comprising a respective set of colors and being associated in the memory with a respective natural language description, each of the concept color palettes comprising a same number of colors;

memory which stores instructions for;

retrieving one of the concept color palettes from the memory corresponding to a user-selected concept, the concept color palette comprising a first set of colors,

receiving a user-selected input image and computing an image color palette for the input image, the image color palette comprising a second set of colors,

mapping colors of the image color palette to colors of the concept color palette to identify, for each of the colors of the image color palette, a corresponding color in the concept color palette,

computing a transformation based on the mapping, and

for pixels of the input image, computing modified color values based on the computed transformation to transform the input image; and

a processor in communication with the memory which executes the instructions.

24. The apparatus of claim 23, further comprising a graphical user interface in communication with the processor for displaying the retrieved concept color palette, modified image, and optionally a modified image color palette.

25. The image adjustment apparatus of claim 24, wherein the graphical user interface is further configured for displaying a selector whereby a user selects a degree of transfer, whereby the transformation is adjustable.

26. The apparatus of claim 23, further comprising a palette training component which learns each of the set of the concept color palettes from a respective set of predefined color palettes.