Image processing of DCT-based video sequences in compressed domain

• US 20060285587A1
• Filed: 06/21/2005
• Published: 12/21/2006
• Est. Priority Date: 06/21/2005
• Status: Active Grant

• Current Assignee: Conversant Wireless Licensing S.à.r.l. (f/k/a Core Wireless Licensing S.a.r.l.) (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Original Assignee: Nokia Corporation
• Inventors: Islam, Asad, Luo, Jiangcong, Chebil, Fehmi
• Granted Patent: US 7,760,808 B2
• Time in Patent Office: Days
• US Class Current: 375/240.200
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20052 Discrete cosine transform [DCT]
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence

Abstract

This invention relates to a method for applying at least one of contrast adjustment and brightness adjustment to a compressed, motion compensated DCT-based video sequence corresponding to an image processing operation, comprising the steps of providing the compressed, motion compensated DCT-based video sequence, applying the image processing operation on the video sequence in compressed domain resulting in an image processed, compressed video sequence, wherein the applying is executed by adjusting of DCT-components defining the DCT-based video sequence.

Claims

1. A method for applying at least one of contrast adjustment and brightness adjustment to a compressed, motion compensated DCT-based video sequence corresponding to an image processing operation, comprising the steps of:

providing said compressed, motion compensated DCT-based video sequence; and

applying said image processing operation on said video sequence in compressed domain resulting in an image processed, compressed video sequence, wherein said applying is executed by adjusting DCT-components defining said DCT-based video sequence.

2. The method according to claim 1, wherein said adjusting of DCT-components is provided by partial decoding and encoding said DCT-based video sequence.

3. The method according to claim 1, wherein shadow effect compensation is provided on the basis of said applying of said image processing of said DCT-based video sequence.

4. The method according to claim 3, wherein said shadow compensation is provided by compensating residuals when out-of-range pixels are detected in a previous frame.

5. The method according to claim 4, wherein said out-of-range pixels are detected in the DCT-based video sequence by setting thresholds for DC and AC coefficients.

6. The method according to claim 4, wherein said shadow effect compensation is provided by partially decoding of certain blocks of the frame, where a possibility for out-of-bound coefficients exists.

7. The method according to claim 1, wherein said video sequence is a digital H.263 bit-stream representing said sequence, wherein the H.263 bit-stream is VLC decoded, DCT coefficients in the bit-stream are de-quantized, and the image processing operations are applied to the DCT coefficients.

8. The method according to claim 1, wherein an amount of change in pixel intensity is exactly the same amount of change in DC coefficient by applying said brightness adjustment of said sequence for intra-coded macroblocks if said sequence corresponds to said H.263 bit-stream.

9. The method according to claim 1, wherein said video sequence is a digital MPEG-4 bit-stream representing said sequence, wherein the MPEG-4 bit-stream is VLC decoded, DCT coefficients in the bit-stream are de-quantised, and the image processing operations are applied to the DCT coefficients.

10. The method according to claim 1, wherein said contrast adjustment in said sequence represented by said MPEG-4 bit-stream is provided as a one-pass process for intra-coded macroblocks.

11. A computer program product for applying at least one of contrast adjustment and brightness adjustment to a compressed, motion compensated DCT-based video sequence relating to a defined environment, comprising program code sections stored on a readable medium for carrying out the steps of claim 1, when said program code sections are run on a computer, a microprocessor based device, a terminal, a network device, a mobile terminal, or a portable communication enabled terminal.

12. A computer program product for image processing of a DCT-based video sequence relating to a defined environment, comprising program code sections stored on a machine-readable medium for carrying out the steps of claim 1, when said program product is run on a computer, a microprocessor based device, a terminal, a network device, a mobile terminal, or a portable communication enabled terminal.

13. A software tool for applying at least one of contrast adjustment and brightness adjustment to a compressed, motion compensated DCT-based video sequence relating to a defined environment, comprising program portions for carrying out the operations of claim 1, when said program portions are implemented in a computer program for execution on a microprocessor based device, processing device, a terminal device, a network device, a mobile terminal, or a portable communication enabled terminal.

14. A module adapted for applying at least one of contrast adjustment and brightness adjustment to a compressed, motion compensated DCT-based video sequence corresponding to an image processing operation, comprising:

a component for providing said compressed DCT-based video sequence; and

a component for applying said image processing operation on said video sequence in compressed domain resulting to an image processed, compressed video sequence, wherein said applying is executed by adjusting of DCT-components defining said DCT-based video sequence.

15. The module according to claim 14, wherein said contrast adjustment is based on the following expression:

$\begin{array}{cc}{Y}^{\mathrm{\prime <br><br>}}<br><br>\left(n,m\right)=\{\begin{array}{cc}\mathrm{\lambda <br><br>}·<br><br>Y<br><br>\left(n,m\right)+\left(\mathrm{\lambda <br><br>}-1\right)·<br><br>\stackrel{\_}{Y_{\mathrm{DC}}}& m=n=0\\ \mathrm{\lambda <br><br>}·<br><br>Y<br><br>\left(n,m\right)& m\ne <br><br>0<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{or}<br><br>\text{\hspace{1em}<br><br>}<br><br>n\ne <br><br>0\end{array}& \left(7\right)\end{array}$

where;

m,n are frequency domain coordinates;

Y′

(n,m) is the DCT transformation of the edited image;

λ

is a contrast scaling factor;

Y(n,m) is the DCT transformation of the original image; and

{overscore (Y_DC)} is the average of the DC coefficients of all blocks within the image.

16. The module according to claim 14, wherein said brightness adjustment is based on the following expression:

$Y_q^{\mathrm{\prime <br><br>}}<br><br>\left(n,m\right)=\{\begin{array}{cc}{Y}_q<br><br>\left(n,m\right)+K^{\mathrm{\prime <br><br>}};<br><br>& m=n=0\\ Y_q<br><br>\left(n,m\right);<br><br>& \mathrm{else}\end{array}$

wherein K′

corresponds to the value of the brightness adjustment in the compressed domain and it is given by;

$\begin{array}{cc}{K}^{\mathrm{\prime <br><br>}}=\{\begin{array}{cc}8·<br><br>K/Q_{\mathrm{DC}};<br><br>& m=n=0\\ 0;<br><br>& \mathrm{else}\end{array}& \left(5\right)\end{array}$

where;

m,n are frequency domain coordinates;

Y_q′

(n,m) is the DCT transformation of the edited image;

Y_q(n,m) is the DCT transformation of the original image; and

Q_DC is the quantization scale for DC coefficient.

17. The module according to claim 14, wherein said video sequence is a digital MPEG-4 bit-stream representing said sequence, wherein the MPEG-4 bit-stream is VLC decoded, DCT coefficients in the bit-stream are de-quantised, and the image processing operations are applied to the DCT coefficients.

18. The module according to claim 14, wherein said contrast adjustment in said sequence represented by said MPEG-4 bit-stream is provided as one-pass process for intra-coded macroblocks.

19. An electronic device adapted for applying at least one of contrast adjustment and brightness adjustment to a compressed, motion compensated DCT-based video sequence corresponding to an image processing operation, comprising:

at least one module for image processing according to claim 14;

an I/O interface;

a memory unit; and

a processor adapted to control all entities within said electronic device.