Methods and apparatus for detecting scene conditions likely to cause prediction errors in reduced resolution video decoders and for using the detected information

• US 6,061,400 A
• Filed: 11/20/1997
• Issued: 05/09/2000
• Est. Priority Date: 11/20/1997
• Status: Expired due to Fees

• Current Assignee: Wells Lamont Corporation (Berkshire Hathaway Incorporated)
• Original Assignee: Hitachi America Limited (Hitachi, Ltd.)
• Inventors: Pearlstein, Larry, Henderson, John, Fuhrer, Jack
• Primary Examiner(s): Britton, Howard
• Assistant Examiner(s): VO, TUNG T
• Application Number:
  US08/974,907
• Time in Patent Office: 901 Days
• Field of Search: 348/390, 348/391, 348/392, 348/397, 348/402, 348/565, 348/566, 348/567, 348/568, 348/423, 348/424, 348/416, 348/555, 348/556, 348/407, 382/118, 382/232, 382/239, 382/233, 382/240
• US Class Current: 375/240
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

Techniques for identifying blocks of pixels, referred to as constant block regions, having approximately the same intensity in terms of luminance values, are discussed. High contrast vertical and/or horizontal edges will cause significant prediction errors in images generated by reduced resolution decoders under certain conditions. Methods for assessing when such conditions exist and a significant prediction error is likely to occur are described. In addition methods and apparatus for minimizing the effect of such prediction errors in downsampling decoders are also described. One specific embodiment is directed to a new video decoder which decodes portions of a single image, e.g., frame, at different resolutions. Areas of the image along high contrast vertical or horizontal edges are decoded at full resolution while other portions of the same image are decoded at reduced resolution. By decoding and storing portions of reduced resolution images at full resolution for reference purposes, the risk of prediction errors resulting from the use of downsampling on reference frames is reduced.

Claims

1. A method of processing encoded video data, comprising the steps of:

receiving encoded video data representing a first image;

performing a decoding operation on the encoded video data to produce a first set of decoded video data representing the first image;

examining the first set of decoded video data to detect the location of constant block regions formed by a plurality of contiguous pixels which have consistent luminance values; and

storing the decoded video data representing the first image.

2. The method of claim 1, wherein the step of performing a decoding operation includes the step of performing a data reduction operation.

3. The method of claim 2, further comprising the steps of:

generating image information which includes information about constant block regions in the first set of decoded video data; and

storing the generated image information.

4. The method of claim 3, wherein the step of examining the first set of decoded image data includes the step of:

identifying the location of detected constant block regions.

5. The method of claim 1, wherein the step of examining the decoded video data to detect the presence and location of constant block regions includes the steps of:

calculating a first mean and a first standard deviation for the luminance values of pixels included in a segment of a first row of pixels; and

determining if the calculated first deviation is below a first preselected threshold value.

6. The method of claim 5, wherein the step of examining the decoded video data further includes the steps of:

calculating a second mean and a second standard deviation for the luminance values of pixels included in a segment of a second row of pixels; and

determining if the calculated second deviation is below a second preselected threshold value.

7. The method of claim 6,

wherein the first and second preselected threshold values are the same, and

wherein the step of examining the first set of decoded video data further includes the steps of:

comparing the first and second means if it is determined that the first and second deviations are below the preselected threshold value.

8. The method of claim 1, further comprising the step of:

receiving encoded video data representing a second image; and

decoding the encoded video data representing the second image as a function of the stored image information and the stored decoded video data representing the first image.

9. The method of claim 8, further comprising the step of:

performing a downsampling operation on the first set of decoded video data prior to storing the first set of decoded video data.

10. The method of claim 9, further comprising the step of:

assessing, as a function of the stored image information, the probability that a predication error which will produce an erroneous vertical or horizontal line in the second decoded image will occur.

11. The method of claim 8, wherein the step of decoding the encoded video data representing the second image generates a second set of decoded video data, the method further comprising the steps of:

performing a downsampling operation on the first set of decoded video data prior to storing the first set of decoded video data;

performing a downsampling operation on at least a portion of the second set of decoded video data; and

storing the second set of decoded video data in a storage device.

12. The method of claim 1, wherein the step of examining the first set of decoded video data includes the step of:

examining display vertical size information included in the received encoded video data representing the first image.

13. The method of claim 3, wherein the step of storing frame information includes the step of:

storing information identifying the location of a detected constant block region.

14. The method of claim 3, wherein the step of storing frame information includes the step of:

storing information identifying the location of an edge of a detected constant block region.

15. A method of processing encoded video data representing a series of frames, comprising the steps of:

receiving encoded video data representing a first frame;

decoding the encoded video data representing the first frame;

analyzing at least a portion of the decoded video data representing the first frame to detect the location of one or more blocks of pixels in the first frame having a consistent intensity;

decoding subsequent video data representing subsequent frames;

analyzing at least a portion of some of the subsequent decoded frames to detect the location of one or more blocks of pixels of consistent intensity in the analyzed subsequent frames; and

comparing the location of blocks of pixels of consistent intensity detected in multiple frames to detect blocks of pixels of consistent intensity which occur in the same location in multiple frames.

16. The method of claim 15, further comprising the step of:

decoding encoded video data including motion vectors representing an additional frame as a function of the location of blocks of pixels of consistent intensity which occur in the same location in multiple frames.

17. The method of claim 16, further comprising the step of:

performing a downsampling operation on at least some of the decoded video data generated by decoding encoded video data; and

storing the downsampled decoded video data for use as reference data.

18. The method of claim 16,

wherein the step of analyzing at least a portion of the decoded video data representing the first frame includes the step of:

analyzing two adjacent rows of pixels to determine if they correspond to a black border.

19. The method of claim 18, wherein the step of decoding encoded video data including motion vectors representing an additional frame includes the steps of:

using the constant block information obtained by comparing the location of blocks of pixels of consistent intensity in previously decoded frames to determine the occurrence of prediction errors during decoding; and

performing a prediction error filtering operation on the decoded video data to reduce the

effect of prediction errors whose occurrence has been determined.

20. A reduced resolution decoding method, comprising the steps of:

decoding a first image encoded at a first resolution;

analyzing the decoded first image to detect black borders, having horizontal or vertical edges, included in the decoded first image;

storing a representation of the first image in a memory device at a resolution lower than the first resolution; and

decoding a second image encoded at the first resolution, the step of decoding the second image including the step of;

performing a motion compensated prediction operation using the stored representation of the first image as reference data and using information regarding black borders included in the decoded first image.

21. An apparatus for processing digital video, comprising:

an inverse discrete cosine transform circuit for receiving and processing transform coded data video data representing video frames;

a constant image region detection circuit coupled to the inverse discrete cosine transform circuit for analyzing data representing video frames to detect a region within at least one frame having a consistent intensity; and

a motion compensated prediction module coupled to the constant image region detection circuit for performing motion compensated prediction operations as a function of a decoded reference frame and data output by the constant image region detection circuit.

22. The apparatus of claim 21, further comprising:

a memory device for storing information identifying the position of a constant image region in the at least one frame.

23. The apparatus of claim 22, further comprising:

a downsampler coupled to an output of the inverse discrete cosine transform circuit for performing a downsampling operation on video data output by the inverse discrete cosine transform circuit.

24. The apparatus of claim 23, wherein the apparatus is a digital television receiver, the apparatus further comprising:

a display device coupled to the memory.

25. A video decoder, comprising:

an inverse discrete cosine transform circuit for receiving and processing transform coded video data representing video frames;

a horizontal edge detection circuit for detecting edges within said video frames;

a controllable data reduction circuit coupled to the inverse discrete cosine transform circuit and to an output of said horizontal edge detection circuit, the controllable data reduction circuit for reducing, as a function of edge information, the amount of data used to represent video frames;

a frame memory coupled to the data reduction device for storing decoded video data representing video frames; and

a motion compensated prediction module for performing motion compensated prediction operations coupled to the frame memory.

26. The video decoder circuit of claim 25,

wherein the data reduction device is a downsampling circuit;

and

wherein the video decoder circuit further comprises;

a reference frame information memory coupled to the horizontal edge detection circuit and the motion compensated prediction module for storing horizontal edge information.

27. The method of claim 25, further comprising:

a resolution control module for coupling said output of said horizontal edge detection circuit to said controllable data reduction circuit.