COLOR CONVERTING IMAGES

• US 20120051636A1
• Filed: 06/17/2011
• Published: 03/01/2012
• Est. Priority Date: 08/31/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Swen, Steve, Bushell, John S., Greenebaum, Kenneth I.
• Granted Patent: US 8,600,159 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85

Abstract

Some embodiments of the invention provide novel methods for converting the color region of images. For instance, the method of some embodiments converts the color of an image from one color region (e.g., from a Rec. 601 color region) to another color region (e.g., to a Rec. 709 color region). In some embodiments, the method receives the image in a first color format that is non-linearly encoded within a first color region. Without performing a linearization operation to linearize the color component values of the image, the method then converts the image into a second color format that is defined with respect to a second color region. In some embodiments, the color-converted image is non-linearly encoded in the second color region.

Claims

1. A method of generating a transform matrix for converting color component values from a first color region to a second color region, the method comprising:

identifying a plurality of color component value sets in the first color region;

for each identified color component value set in the first color region, generating a desired color component value set in the second color region; and

using the identified and generated color component value sets to identify an n-by-m transform matrix for converting color component values from the first color region to the second color region.

2. The method of claim 1, wherein using the color component value sets comprises examining different matrix parameters for the n-by-m transform matrix in order to identify a set of matrix parameters that from the identified color component value sets, produces a plurality of color component value sets in the second color region that closely match the desired color component value sets in the second color region.

3. The method of claim 2 further comprising terminating the examination of different matrix parameters after identifying a set of matrix parameters that produces a plurality of color component value sets that sufficiently match the desired color component value sets.

4. The method of claim 2 further comprising terminating the examination of different matrix parameters after identifying a set of matrix parameters that produces a plurality of color component value sets that best match the desired color component value sets after a certain duration of the examination.

5. The method of claim 4, wherein the duration is a temporal duration.

6. The method of claim 4, wherein the duration is based on a number of matrix parameter sets that are examined during the examination.

7. The method of claim 2, wherein the plurality of color component value sets produced by the identified set of matrix parameters result in a difference between the desired and produced color component value sets that is sufficiently minimal.

8. The method of claim 1, wherein using the color component value sets comprises examining different matrix parameters for the n-by-m transform matrix in order to identify a set of matrix parameters that from the identified color component value sets, produces a plurality of color component value sets in the second color region with minimal differences with the desired color component value sets in the second color region.

9. The method of claim 8, wherein identifying the plurality of color component value sets comprises identifying more sets in a sub-region of interest in the first color region than sets in a sub-region of the first color region that is not of interest.

10. The method of claim 9, wherein the sub-region of interest includes color values whose poor replication in the second color region will be perceivable by a viewer.

11. The method of claim 9, wherein the sub-region of interest includes color values related to one of skin tones, foliage, and sky.

12. The method of claim 1, wherein the n-by-m matrix is a first identified matrix, the method further comprising:

using the identified and generated color component value sets to identify a second n-by-m transform matrix that converts color component values from the first color region to the second color region,

wherein said first and second transform matrices differ in that the first transform matrix is biased for converting color component values in a first sub-region of the first region while the second transform matrix is biased for converting color component values in a second sub-region of the first region.

13. The method of claim 1, wherein n and m are equal integer values.

14. The method of claim 1, wherein n and m are different integer values.

15. The method of claim 1, wherein the identified color component value sets in the first color region are gamma corrected.

16. The method of claim 1,

wherein the color component values in the first color region are defined in a non-linear domain;

and

wherein the identified n-by-m transform matrix is for converting the color component values from the first color region to the second color region without transforming the color component values to a linear domain.

18. The non-transitory machine readable medium of claim 1, wherein the transform matrix is defined by (i) generating a desired color sample in the second color region for each of a plurality of color samples in the first color region, and (ii) examining different parameters for the matrix in order to identify a set of matrix parameters that from the first-region color samples, produces a plurality of color samples in the second color region with minimal differences with the desired color samples in the second color region.

19. The non-transitory machine readable medium of claim 18, wherein the generated, desired color samples are color samples in the second color region that are produced from color samples in the first color region through a set of operations that includes the linearization.

20. The non-transitory machine readable medium of claim 18, wherein the plurality of color samples in the first color region includes more color samples in a sub-region of interest in the first color region than color samples in a sub-region of the first color region that is not of interest.

17. A non-transitory machine readable medium comprising a program for converting color component values from a first color region to a second color region, the program comprising sets of instructions for:

receiving a gamma-corrected image comprising a plurality of pixels, each pixel defined by reference to a color component value set that is specified in the first color region; and

using an n-by-m transform matrix to convert the image'"'"'s color component value sets from the first color region to the second color region without linearization of the color component value sets, said transform matrix defined to reduce errors caused by foregoing the linearization.

21. The non-transitory machine readable medium of claim 17, wherein the program further comprises a set of instructions for selecting the n-by-m transform matrix from a plurality of different n-by-m transform matrices that are differently defined to reduce errors in color converting different sub-regions of the first color region.

22. The non-transitory machine readable medium of claim 21, wherein a first matrix is defined to reduce errors in color converting first region color values relating to skin tones, while a second matrix is defined to reduce errors in color converting first region color values not related to the skin tones.

23. The non-transitory machine readable medium of claim 22, wherein the second matrix is defined to reduce errors in color converting first region color values relating to either foliage or sky.

24. The non-transitory machine readable medium of claim 17, wherein the program further comprises sets of instructions for:

performing a color coordinate transform operation on the color component values in the first color region before using the n-by-m transform matrix that performs the color region conversion operation; and

performing a color coordinate transform operation on the color component values after using the n-by-m transform matrix that performs the color region conversion operation.

25. The non-transitory machine readable medium of claim 17, wherein the n-by-m transform matrix operation not only performs a color region conversion operation but also performs a color coordinate transform operation that converts the color component values of the image from a first color coordinate system to a second color coordinate system.

26. The non-transitory machine readable medium of claim 25, wherein the program further comprises sets of instructions for:

performing a compositing operation on the image after the image has been color coordinate, color region transformed; and

performing a color coordinate transform operation on the image after the compositing operation, in order to transform the image back to the image'"'"'s original color coordinate system.

27. The non-transitory machine readable medium of claim 17,

wherein the color component values include a brightness value,

wherein the n-by-m transform matrix is a first n-by-m transform matrix that is defined for a first brightness range,

wherein the program further comprises sets of instructions for:

identifying for each color component value set in the image, a brightness range in which the color component value set falls;

selecting for each color component value set, an n-by-m transform matrix from a plurality of n-by-m transform matrices based on the set'"'"'s brightness range; and

using the selected n-by-m transform matrix for each color component value set to transform the set from the first color region to the second color region.

28. An apparatus comprising:

an interface for receiving a gamma-corrected image comprising a plurality of pixels, each pixel defined by reference to a color component value set that is specified in a first color region; and

a color processor for using an n-by-m transform matrix to convert the image'"'"'s color component values from the first color region to the second color region without linearization of the color component value sets, said transform matrix defined to reduce errors caused by foregoing the linearization.

29. The apparatus of claim 28, wherein the interface is a network interface for receiving the image over a network.

30. The apparatus of claim 28, wherein the image is part of a video received by the apparatus.

31. The apparatus of claim 28, wherein the interface receives the image in an encoded format, the apparatus further comprising a decoder for decoding the image before the color processor uses the transform matrix.

32. The apparatus of claim 28,

wherein the transform matrix is defined by (i) generating a desired color sample in the second color region for each of a plurality of color samples in the first color region, and (ii) examining different parameters for the matrix in order to identify a set of matrix parameters that from the first region color samples, produces a plurality of color samples in the second color region with minimal differences with the desired color samples in the second color region,

wherein the generated, desired color samples are color samples in the second color region that are produced from color samples in the first color region through a set of operations that includes the linearization.

33. The apparatus of claim 28, wherein the color processor is a first color processor, wherein the apparatus further comprises:

a second color processor for performing a color coordinate transform operation on the color component values in the first color region before using the n-by-m transform matrix to perform the color region conversion operation; and

a third color processor for performing a color coordinate transform operation on the color component values in the first color region after using the n-by-m transform matrix to perform the color region conversion operation.

34. The apparatus of claim 28, wherein the color processor not only performs a color region conversion operation but also performs a color coordinate transform operation that converts the color component values of the image from a first color coordinate system to a second color coordinate system.

35. The apparatus of claim 34, wherein the color processor is a first color processor, the apparatus further comprising:

a compositor for performing a compositing operation on the image after the image has been color coordinate, color region transformed; and

a second color processor for performing a color coordinate transform operation on the image after the compositing operation, in order to transform the image back to the image'"'"'s original color coordinate system.

36. A non-transitory machine readable medium comprising a program for converting color component values from a first color region to a second color region, the program comprising sets of instructions for:

receiving an image comprising a plurality of pixels, each pixel defined by reference to a color component value set that is specified in the first color region;

for each pixel, retrieving from a storage structure at least one set of adjustment values for adjusting at least one color component value set that is defined in the first color region to be in a second color region; and

applying the set of adjustment values retrieved for each pixel to the color component value set of the pixel.

37. The non-transitory machine readable medium of claim 36,

wherein the storage structure stores sets of adjustment values for adjusting only a subset of pixel color component value sets that the program may have to convert,

wherein the set of instructions for retrieving comprises a set of instructions for retrieving a plurality of adjustment value sets for a particular pixel,

wherein the set of instructions for applying comprises a set of instructions for calculating one set of adjustment values from the plurality of retrieved adjustment value sets and a set of instructions for applying the calculated set of adjustment values to the color component value set for the particular pixel.

38. The non-transitory machine readable medium of claim 37, wherein the set of instructions for retrieving further comprises a set of instructions for retrieving only one set of adjustment values for another particular pixel when the storage structure stores a single adjustment set that by itself can be used to identify the other particular pixel'"'"'s color component value set.

39. The non-transitory machine readable medium of claim 36, wherein the image is part of a video clip, wherein the storage structure stores adjustment values only for a range of possible video component color values for display.

40. The non-transitory machine readable medium of claim 37, wherein the adjustment values that are stored in the storage structure include more adjustment values for color component value sets in a sub-region of interest of the first color region than color component value sets in a sub-region of the first color region that is not of interest.

41. The non-transitory machine readable medium of claim 37, wherein the color component adjustment values that are stored in the storage structure are biased towards different sub-regions of interest of the first color region.