Method of enhancing images extracted from video

• US 7,760,805 B2
• Filed: 05/31/2005
• Issued: 07/20/2010
• Est. Priority Date: 05/31/2005
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Owens, James
• Primary Examiner(s): Philippe; Gims S
• Application Number:
  US11/142,084
• Publication Number:
  US 20060268981A1
• Time in Patent Office: 1,876 Days
• Field of Search: 375/240.12, 375/240.13, 375/240.29, 375/240.02, 375/240.27, 382/269, 382/274, 382/275, 382/236
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

Method and devices for enhancing still digital images extracted from a compressed video sequences are disclosed herein, wherein the compressed video sequence comprises at least one I-frame and at least one P-frame. The method comprises determining whether an I-frame or a P-frame of video has been extracted, determining the quantization matrix of the previous I-frame if a P-frame has been extracted, and reconstructing the extracted video frame. The image is enhanced based on the quantization matrix of the present I-frame if the extracted frame is an I-frame and based on the quantization matrix of the previous I-frame if a P-frame has been extracted.

Claims

1. A method of enhancing a still digital image extracted from a compressed video sequence, said compressed video sequence comprising at least one I-frame and at least one P-frame, said method comprising:

determining whether an I-frame or a P-frame of video has been extracted;

determining the quantization matrix of the previous I-frame if a P-frame has been extracted;

reconstructing the extracted video frame

enhancing the image based on the quantization matrix of the present I-frame if the extracted frame is an I-frame and enhancing the image based on the quantization matrix of the previous I-frame if a P-frame has been extracted.

2. The method of claim 1, wherein the enhancing further comprises:

computing spatially-shifted forward transforms on the reconstructed video frame to generate respective sets of forward transform coefficients;

applying nonlinear transforms to the forward transform coefficients of each set;

computing inverse transforms of the sets of nonlinearly transformed forward transform coefficients to generate respective intermediate images;

computing an output image with pixel values based at least in part on the intermediate images.

3. The method of claim 2, wherein the computed spatially-shifted forward transforms are block transforms.

4. The method of claim 3, wherein the block transforms are two-dimensional block discrete cosine transforms.

5. The method of claim 2, wherein the forward transform coefficients are nonlinearly transformed by setting to zero each coefficient with an absolute value less than a first value and leaving unchanged each coefficient with an absolute equal to at least a second value.

6. The method of claim 5, further comprising sharpening the forward transform coefficients by increasing nonlinear transform parameters by respective factors that are greater for higher spatial frequency forward transform coefficients than for lower spatial frequency forward transform coefficients.

7. The method of claim 2, wherein computing the output image comprises computing a base image from a combination of intermediate images.

8. The method of claim 7, wherein the base image has pixel values corresponding to respective averages of values of corresponding pixels in the intermediate images.

9. The method of claim 7, wherein computing the output image further comprises computing a ringing correction image based at least in part on the computed measures of local spatial intensity variability.

10. The method of claim 9, wherein each pixel in the ringing correction image is associated with a corresponding pixel in each of the intermediate images.

11. The method of claim 10, further comprising assigning to each pixel in the ringing correction image a value of a corresponding intermediate image pixel having a lowest computed measure of local spatial intensity variability of the corresponding intermediate image pixels.

12. The method of claim 10, further comprising assigning to each pixel in the ringing correction image a value corresponding to an average of multiple corresponding intermediate image pixels in a lowest percentile of local spatial variability measures of the corresponding intermediate image pixels.

13. The method of claim 9, wherein the output image is computed by combining pixel values from the base image and the ringing correction image.

14. The method of claim 13, wherein the output image is computed by a weighted combination of the base image and the ringing correction image.

15. The method of claim 14, further comprising identifying transition regions in the base image.

16. The method of claim 15, wherein identifying transition regions in the base image comprises, for a given pixel in the base image, computing a measure of local contrast for a local region encompassing the given pixel and computing a measure of block contrast for a larger region of the base image encompassing the local region.

17. The method of claim 16, wherein the measures of local contrast and block contrast correspond to pixel ranges computed based on two concentric windows superimposed over pixels of the base image.

18. The method of claim 16, further comprising mapping the measures of local contrast and block contrast to weights corresponding to respective pixel contributions of the base image and the ringing correction image to pixels of the output image.

19. The method of claim 18, wherein the base image weight is 1-α

, the ringing correction image weight is α

, and α

has a value ranging from 0 to 1 that is larger for pixels with relatively high ratios of block contrast to local contrast and is smaller for pixels with relatively low ratios of block contrast to local contrast.

20. The method of claim 19, wherein

$\alpha$

⁡

( Δ

) = 1 1 + ⅇ

- ( Δ

- T ) a

wherein Δ

=block contrast—

local contrast, and a and T are variable parameters.

21. The method of claim 14, wherein the base image contribution to the output image is greater than the ringing correction image contribution for pixels remote from transition regions in the base image.

22. The method of claim 14, wherein the base image contribution to the output image is greater than the ringing correction image contribution for pixels in transition regions in the base image.

23. The method of claim 14, wherein the base image contribution to the output image is less than the ringing correction image contribution for pixels adjacent to transition regions in the base image.