Color matching using color segmentation

• US 8,594,426 B2
• Filed: 02/04/2011
• Issued: 11/26/2013
• Est. Priority Date: 02/04/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bryant, Andrew, Pettigrew, Daniel, Fedkiw, Olivier
• Primary Examiner(s): Liew, Alex
• Application Number:
  US13/021,694
• Publication Number:
  US 20120201451A1
• Time in Patent Office: 1,026 Days
• Field of Search: 382/103, 382162-167, 382173-180
• US Class Current: 382/173
• CPC Class Codes: H04N 1/60 Colour correction or control H04N1/54 takes precedence
• CPC Class Codes: H04N 1/6075 Corrections to the hue
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only

Abstract

Some embodiments provide a computer program for matching colors of a first image to colors of a second image. The computer program identifies a set of transforms for converting the first image to a device-independent color space. The computer program modifies the set of transforms to segment colors of the first image into a set of segmented colors. The computer program applies the modified set of transforms to pixel values of the first image in order to segment the pixels of the first image into the set of segmented colors. The computer program applies the modified set of transforms to pixel values of the second image in order to segment the pixels of the second image into the set of segmented colors. The computer program matches each segmented color in the set of segmented colors in the first image to the segmented color in the second image.

Claims

1. A non-transitory computer readable medium storing a computer program which when executed by at least one processing unit matches colors of a first image to colors of a second image, the program comprising sets of instructions for:

identifying a first set of transforms for converting pixel values of the first and second images from first and second color spaces to a device-independent color space;

modifying the first set of transforms to generate a second set of transforms that not only convert the pixel values of the first and second images to the device-independent color space, but also shift colors during the transform in order to segment the colors into more-defined color segments in the device-independent color space;

applying the second set of transforms to pixel values of the first and second images; and

modifying, for each color segment in the color segments, pixel values in the first image so that colors of pixels in the first image that fall within the color segment shift towards colors of pixels in the second image that fall within the color segment.

2. The non-transitory computer readable medium of claim 1, wherein the second set of transforms includes a transform for segmenting highlight colors into blue colors.

3. The non-transitory computer readable medium of claim 2, wherein highlight colors include colors having a luminance value greater than a threshold value.

4. The non-transitory computer readable medium of claim 1, wherein the second set of transforms includes a transform for segmenting shadow colors into brown or red colors.

5. The non-transitory computer readable medium of claim 4, wherein shadow colors include colors having a luminance value less than a threshold value.

6. The non-transitory computer readable medium of claim 1, wherein the device-independent color space is an XYZ color space.

7. The non-transitory computer readable medium of claim 1, wherein the set of instructions for modifying pixel values in the first image comprises a set of instructions for modifying pixel values in the first image so that, for each color segment in the color segments, a set of characteristics of the colors of pixels in the first image that fall within the color segment matches the set of characteristics of the colors of pixels in the second image that fall within the color segment.

8. The non-transitory computer readable medium of claim 7, wherein the set of characteristics includes an average color of pixels in the segmented color.

9. The non-transitory computer readable medium of claim 7, wherein the set of characteristics includes an average color of dark pixels in the segmented color.

10. The non-transitory computer readable medium of claim 7, wherein the set of characteristics includes an average color of bright pixels in the segmented color.

11. The non-transitory computer readable medium of claim 7, wherein the set of characteristics includes an average saturation of pixels in the segmented color.

12. The non-transitory computer readable medium of claim 7, wherein the set of characteristics includes an average contrast of pixels in the segmented color.

13. The non-transitory computer readable medium of claim 1, wherein the first image is a frame of a video clip.

14. The non-transitory computer readable medium of claim 1, wherein the video clip is a first video clip, wherein the second image is a frame of a second video clip.

15. The non-transitory computer readable medium of claim 1, wherein the first image is a still image.

16. A method for color matching a first image to a second image, the method comprising:

determining a first set of transforms for modifying pixel values of pixels in the first image to match a first set of characteristics of the first image to the first set of characteristics of the second image;

segmenting pixels of the first image into a set of colors and pixels of the second image into the set of colors;

determining a second set of transforms for modifying pixel values of pixels in the first image to match a second set of characteristics of each color in the set of colors of the first image to the second set of characteristics of the corresponding color of the second image; and

modifying the pixel values of the first image to match the pixel values of the second image by applying the first set of transforms and the second set of transforms to the pixel values of the first image so that the first set of characteristics of the first image matches to the first set of characteristics of the second image and the second set of characteristics of each color in the set of colors of the first image matches to the second set of characteristics of the corresponding color of the second image.

17. The method of claim 16, wherein segmenting the pixels comprises:

applying to the pixels of the first image a third set of transforms for mapping the pixels of the first image into the set of colors; and

applying to the pixels of the second image a fourth set of transforms for mapping the pixels of the second image into the set of colors.

18. The method of claim 17, wherein the third set of transforms and the fourth set of transforms are the same.

19. The method of claim 17, wherein the third set of transforms is further for converting the first image to a device-independent color space.

20. The method of claim 19, wherein the fourth set of transforms is further for converting the second image to a device-independent color space.

21. The method of claim 20, wherein the device-independent color space is an XYZ color space.

22. The method of claim 16, wherein the modification of the pixel values of the first image by applying the first set of transforms to the first image is performed before the segmentation of colors of the first and second images, the determination of the second set of transforms, and the modification of the pixel values of the first image by applying the second set of transforms.

23. The method of claim 16, wherein the first and second sets of characteristics each includes average color.

24. The method of claim 16, wherein the first and second sets of characteristics each includes average color of bright pixels.

25. The method of claim 16, wherein the first and second sets of characteristics each includes average color of dark pixels.

26. The method of claim 16, wherein the first and second sets of characteristics each includes average saturation.

27. The method of claim 16, wherein the first and second sets of characteristics each includes average contrast.