Dominant color extraction using perceptual rules to produce ambient light derived from video content

• US 7,894,000 B2
• Filed: 06/27/2005
• Issued: 02/22/2011
• Est. Priority Date: 06/30/2004
• Status: Expired due to Fees

• Current Assignee: TP Vision Holding BV (TPV Technology Co., Ltd. (f/k/a Nanjing Huadong Electronics Information & Technology))
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Gutta, Srinivas, Diederiks, Elmo M. A., Elting, Mark J.
• Primary Examiner(s): LEE, MICHAEL
• Application Number:
  US11/571,283
• Publication Number:
  US 20070242162A1
• Time in Patent Office: 2,066 Days
• Field of Search: 348/663, 348/603, 348/602, 348/739, 348/553, 348/659, 348/655, 348/656, 348/563, 348/708
• US Class Current: 348/603
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 9/12 Picture reproducers H04N9/11 takes precedence
• CPC Class Codes: H04N 9/64 Circuits for processing colour signals H04N9/77 takes precedence; camera processing pipelines for processing colour signals H04N23/84
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88
• CPC Class Codes: H05B 47/155 Coordinated control of two or more light sources
• CPC Class Codes: Y02B 20/40 Control techniques providing energy savings, e.g. smart controller or presence detection

Abstract

Extracting and processing video content encoded in a rendered color space to be emulated by an ambient light source, using perceptual rules for intelligent dominant color selection. Steps include quantizing the video color space; performing dominant color extraction by using a mode, median, mean, or weighted average of pixel chromaticities; applying perceptual rules to further derive dominant chromaticities via [1] chromaticity transforms; [2] a weighted average using a pixel weighting function influenced by scene content; and [3] extended dominant color extraction where pixel weighting is reduced for majority pixels; and [4] transforming the dominant color chosen to the ambient light color space using tristimulus matrices. A color of interest can be further analyzed to produce a true dominant color, and past video frames can guide selection of dominant colors in future frames.

Claims

1. A method for dominant color extraction from video content encoded in a rendered color space (RGB) to produce, using perceptual rules, a dominant color (DC) for emulation by an ambient light source (88), comprising:

[1] Performing dominant color extraction from pixel chromaticities (Cp) from said video content in said rendered color space to produce a dominant color by extracting any of;

[a] a mode of said pixel chromaticities;

[b] a median of said pixel chromaticities;

[c] a weighted average by chromaticity of said pixel chromaticities;

[d] a weighted average of said pixel chromaticities using a pixel weighting function (W) that is a function of any of pixel position (i, j), chromaticity (x, y, R), and luminance (L);

[2] Further deriving the chromaticity of said dominant color in accordance with a perceptual rule, said perceptual rule chosen from any of;

[a] a simple chromaticity transform (SCT);

[b] a weighted average using said pixel weighting function (PF8) so further formulated as to exhibit an influence from scene content that is obtained by assessing any of chromaticity and luminance for a plurality of pixels in said video content;

[c] an extended dominant color extraction (EE8) using a weighted average where said pixel weighting function is formulated as a function of scene content that is obtained by assessing any of chromaticity and luminance for a plurality of pixels in said video content, with said pixel weighting function further formulated such that weighting is at least reduced for majority pixels (MP); and

[3] Transforming said dominant color from said rendered color space to a second rendered color space (R′

G′

B′

) so formed as to allow driving said ambient light source.

2. The method of claim 1, wherein the simple chromaticity transform chooses a chromaticity found in said second rendered color space.

3. The method of claim 1, wherein said pixel weighting function is so formulated to provide darkness support by:

[4] assessing said video con tent to establish that a scene brightness in said scene content is low; and

[5] performing any of;

[a] using said pixel weighting function so further formulated to reduce weighting of bright pixels; and

[b] broadcasting a dominant color obtained using reduced luminance relative to that which would otherwise be produced.

4. The method of claim 1, wherein said pixel weighting function is so formulated to provide color support by [6] assessing said video content to establish that a scene brightness in said scene content is high;

and [7] performing any of;

[a] using said pixel weighting function so further formulated to reduce weighting of bright pixels; and

[b] performing step [2][c].

5. The method of claim 1, wherein said extended dominant color extraction is repeated individually for different scene features (J8, V111, V999) in said video content, forming a plurality of dominant colors (DC1, DC2, DC3) and step [1] is repeated where each of said plurality of dominant colors is designated as a pixel chromaticity.

6. The method of claim 1, wherein after said extended dominant color extraction is performed, step [1] is repeated separately for pixel chromaticities in a newly appearing scene feature (V999).

7. The method of claim 1, wherein said method comprises, prior to step [1], quantizing at least some pixel chromaticities (Cp) from said video content in said rendered color space to form a distribution of assigned colors (AC), and during step [1], obtaining at least some of said pixel chromaticities from said distribution of assigned colors.

8. The method of claim 7, wherein said quantizing comprises binning said pixel chromaticities into at least one superpixel (XP).

9. The method of claim 7, wherein at least one of said assigned colors is a regional color vector (V) that is not necessarily in said rendered color space.

10. The method of claim 9, wherein said regional color vector lies in said second rendered color space.

11. The method of claim 7, additionally comprising establishing at least one color of interest (COI) in said distribution of assigned colors and extracting pixel chromaticities assigned thereto to derive a true dominant color (TDC) to be designated as said dominant color.

12. The method of claim 1, wherein said dominant color comprises a pallet of dominant colors (DC1+DC2+DC3), each derived from applying said method.

13. A method for dominant color extraction from video content encoded in a rendered color space (RGB) to produce, using perceptual rules, a dominant color (DC) for emulation by an ambient light source (88), comprising:

[0] Quantizing at least some pixel chromaticities (Cp) from said video content in said rendered color space to form a distribution of assigned colors (AC);

[1] Performing dominant color extraction from said distribution of assigned colors to produce a dominant color by extracting any of;

[a] a mode of said distribution of assigned colors;

[b] a median of said distribution of assigned colors;

[c] a weighted average by chromaticity of said distribution of assigned colors;

[d] a weighted average of said distribution of assigned colors using a pixel weighting function (W) that is a function of any of pixel position (i, j), chromaticity (x, y, R), and luminance (L);

[2] Further deriving the chromaticity of said dominant color in accordance with a perceptual rule, said perceptual rule chosen from any of;

[a] a simple chromaticity transform (SCT);

[b] a weighted average using said pixel weighting function (PF8) so further formulated as to exhibit an influence from scene content that is obtained by assessing any of chromaticity and luminance for a plurality of pixels in said video content;

[c] an extended dominant color extraction (EE8) using a weighted average where said pixel weighting function is formulated as a function of scene content that is obtained by assessing any of chromaticity and luminance for a plurality of pixels in said video content, with said pixel weighting function further formulated such that weighting is at least reduced for assigned colors attributable to majority pixels (MP); and

[3] Transforming said dominant color from said rendered color space to a second rendered color space (R′

G′

B′

) so formed as to allow driving said ambient light source.

14. The method of claim 13, wherein the simple chromaticity transform chooses a chromaticity found in said second rendered color space.

15. The method of claim 13, wherein said pixel weighting function is so formulated to provide darkness support by:

[4] assessing said video content to establish that a scene brightness in said scene content is low; and

[5] performing any of;

[a] using said pixel weighting function so further formulated to reduce weighting of assigned colors attributable to bright pixels; and

[b] broadcasting a dominant color obtained using reduced luminance relative to that which would otherwise be produced.

16. The method of claim 13, wherein said pixel weighting function is so formulated to provide color support by [6] assessing said video content to establish that a scene brightness in said scene content is high;

and [7] performing any of;

[a] using said pixel weighting function so further formulated to reduce weighting of assigned colors attributable to bright pixels; and

[b] performing step [2][c].

17. The method of claim 13, wherein said extended dominant color extraction is repeated individually for different scene features (38, V111, V999) in said video content, forming a plurality of dominant colors (DC1, DC2, DC3) and step [1] is repeated where each of said plurality of dominant colors is designated as an assigned color.

18. The method of claim 13, wherein after said extended dominant color extraction is performed, step [1] is repeated separately for assigned colors corresponding to pixels portraying a newly appearing scene feature (V999).

19. The method of claim 13, additionally comprising establishing at least one color of interest (COI) in said distribution of assigned colors and extracting original pixel chromaticities assigned thereto to derive a true dominant color (TDC) to be designated as said dominant color.

20. A method for dominant color extraction from video content encoded in a rendered color space (RGB) to produce, using perceptual rules, a dominant color (DC) for emulation by an ambient light source (88), comprising:

[0] Decoding said video content in said rendered color space into a plurality of frames, and quantizing at least some pixel chromaticities (Cp) from said video content in said rendered color space to form a distribution of assigned colors (AC);

[1] Performing dominant color extraction from said distribution of assigned colors to produce a dominant color by extracting any of;

[a] a mode of said distribution of assigned colors;

[b] a median of said distribution of assigned colors;

[c] a weighted average by chromaticity of said distribution of assigned colors;

[d] a weighted average of said distribution of assigned colors using a pixel weighting function (W) that is a function of any of pixel position (i, j), chromaticity (x, y, R), and luminance (L);

[2] Further deriving the chromaticity of said dominant color in accordance with a perceptual rule, said perceptual rule chosen from any of;

[a] a simple chromaticity transform (SCT);

[b] a weighted average using said pixel weighting function (PF8) so further formulated as to exhibit an influence from scene content that is obtained by assessing any of chromaticity and luminance for a plurality of pixels in said video content;

[c] an extended dominant color extraction (EE8) using a weighted average where said pixel weighting function is formulated as a function of scene content that is obtained by assessing any of chromaticity and luminance for a plurality of pixels in said video content, with said pixel weighting function further formulated such that weighting is at least reduced for assigned colors attributable to majority pixels (MP); and

[3a] Transforming said dominant color from said rendered color space to an unrendered color space (XYZ);

[3b] Transforming said dominant color from said unrendered color space to said second rendered color space, assisted by

[3c] matrix transformations of primaries (RGB, R′

G′

B′

) of said rendered color space and second rendered color space to said unrendered color space using first and second tristimulus primary matrices (M₁, M₂); and

deriving a transformation of said color information into said second rendered color space (R′

G′

B′

) by matrix multiplication of said primaries of said rendered color space, said first tristimulus matrix, and the inverse of said second tristimulus matrix (M₂)<sup>−

1</sup>.