Apparatus and method for optimizing encoding and performing automated steerable image compression in an image coding system using a perceptual metric

• US 6,529,631 B1
• Filed: 03/28/1997
• Issued: 03/04/2003
• Est. Priority Date: 03/29/1996
• Status: Expired due to Term

• Current Assignee: MediaTek, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Lee, Jungwoo, Peterson, Heidi
• Primary Examiner(s): Johnson, Timothy M.
• Application Number:
  US08/825,474
• Time in Patent Office: 2,167 Days
• Field of Search: 348/404, 348/394, 348/395, 348/400, 348/403, 348/409, 348/415, 382/239, 382/242, 382/243, 382/251, 382/250, 382/248, 382/238, 382/235, 358/430, 358/26.2, 358/467, 358/432, 358/433
• US Class Current: 382/232
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression

Abstract

A method and apparatus for adjusting encoding parameters in accordance with a perceptual metric derived from analyzing the contents of an image or video being coded. Namely, the perceptual metric is produced by comparing an original image (or image sequence) to a reconstructed image (or image sequence) and processing the comparison results using a quantitative perceptual difference measure. The difference measure results in a perceptual metric representing the “fidelity” of the reconstructed image. The metric is used to control the encoding parameters such that the coding of the image can be iteratively adjusted. Furthermore, the method and apparatus is used to provide an automated steerable image compression system.

Claims

1. Apparatus for adjusting an encoding parameter of an image encoder to optimize encoder performance comprising:

a perceptual metric generator for determining a perceptual metric using a just noticeable difference map that represents the fidelity of an encoded image, where said perceptual metric is representative of a prediction as to how a human eye would perceive a reconstructed image of said encoded image; and

an encoder parameter selector, coupled to said perceptual metric generator, for selecting a value for said encoding parameter in response to said perceptual metric.

2. The apparatus of claim 1 wherein said encoding parameter is a quantization scale for the image encoder.

3. The apparatus of claim 1 wherein said perceptual metric generator determines the metric by comparing an input image with a reconstructed image, where the reconstructed image is the input image having been encoded and decoded.

4. The apparatus of claim 1 wherein the perceptual metric generator produces a plurality of metrics for said encoded image to represent fidelity for a plurality of regions of the encoded image, where said plurality of metrics forms a fidelity map.

5. The apparatus of claim 4 further comprising an importance map generator for generating an importance map that defines regions of the image that are to be encoded with higher fidelity than other regions, the importance map is compared to the fidelity map to steer the coding such that certain regions of the image are encoded with a higher fidelity than other regions as defined by the importance map.

6. The apparatus of claim 1 further comprising an image difference detector for detecting when a current input image is substantially different from a previous input image and for interrupting selection of said value of said quantization scale when said current input image is detected as being substantially different from said previous input image.

7. Apparatus for encoding an input image comprising:

an image encoder for encoding said input image and producing an encoded image;

a perceptual metric generator, coupled to said image encoder, for determining a perceptual metric using a just noticeable difference map that represents the fidelity of said encoded image as compared to said input image, where said perceptual metric is representative of a prediction as to how a human eye would perceive a reconstructed image of said encoded image; and

an encoder parameter generator, coupled to said perceptual metric generator, for generating a value for a quantization scale in response to said perceptual metric, where said quantization scale is used by said image encoder to encode said input image.

8. The apparatus of claim 7 wherein said image encoder is a block-based video encoder.

9. The apparatus of claim 8 wherein the perceptual metric generator produces a plurality of metrics for said encoded image to represent fidelity for a plurality of regions of the encoded image, where said plurality of metrics forms a fidelity map.

10. The apparatus of claim 9 further comprising an importance map generator for generating an importance map that defines regions of the input image that are to be encoded with higher fidelity than other regions, the importance map is compared to the fidelity map to steer the coding such that certain regions of the image are encoded with a higher fidelity than other regions as defined by the importance map.

11. The apparatus of claim 7 further comprising an image difference detector for detecting when a current input image is substantially different from a previous input image and for interrupting generation of said value of said quantization scale when said current input image is detected as being substantially different from said previous input image.

12. A method of encoding an image comprising the steps of:

a) encoding an input image using an initial value for an encoding parameter to produce an encoded image;

b) decoding said encoded image to produce a decoded image;

c) comparing said decoded image to said input image;

d) computing, in response to said comparison, a perceptual fidelity metric using a just noticeable difference map for the encoded image. where said perceptual fidelity metric is representative of a prediction as to how a human eye would perceive said decoded image; and

e) altering a value of the encoding parameter in response to said perceptual fidelity metric.

13. The method of claim 12 further comprising the step of:

iterating said steps a) through d) until said fidelity metric meets a predefined fidelity criterion.

14. The method of claim 13 further comprising the steps of:

computing a plurality-of fidelity metrics for a respective plurality of regions of said encoded image, where said plurality of fidelity metrics forms a fidelity map.

15. The method of claim 14 further comprising the steps of:

supplying an importance map that defines regions of said input image to be encoded with higher fidelity than other regions of the input image; and

comparing the importance map with the fidelity map to determine regions to be encoded with higher fidelity using differing encoding parameter values.

16. The method of claim 12 wherein the encoding parameter is a quantization scale.

17. The method of claim 12 wherein the encoding step encodes a plurality of macroblocks representing a sequence of images using a value of the encoding parameter assigned to each macroblock and the computing step produces a fidelity metric for each macroblock.

18. The method of claim 12 wherein the altering step is interrupted when a current input image is substantially different from a previous input image detected in an image sequence.