Macroblock based scaling of images using reduced memory bandwidth

• US 8,872,856 B1
• Filed: 08/03/2009
• Issued: 10/28/2014
• Est. Priority Date: 08/14/2008
• Status: Active Grant

• Current Assignee: NXP USA, Inc. (NXP Semiconductors NV)
• Original Assignee: Zenverge, Inc. (NXP Semiconductors NV)
• Inventors: Masterson, Anthony D.
• Primary Examiner(s): AMIN, JWALANT B
• Application Number:
  US12/534,758
• Time in Patent Office: 1,912 Days
• Field of Search: None
• US Class Current: 345/660
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/4023 based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/0414 Vertical resolution change
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory

Abstract

A method and/or system of scaling images by providing a horizontal macroblock scaler with pixels of the images in units of macroblocks and a vertical macroblock scaler where the pixels of the output image are in units of macroblocks. For the horizontal macroblock scaler macroblocks decoded or vertically scaled are read into the horizontal scaler to generate horizontally scaled macroblocks of pixels. The horizontal scaler need not access data stored in a memory, which reduces the memory bandwidth need for scaling the image. For the vertical macroblock scaler macroblocks decoded or horizontally scaled are read into the vertical scaler from memory to generate vertical scaled macroblocks of pixel. The output of the vertical scaler can be sent directly to an encoder without going through memory first, which reduces the memory bandwidth need for scaling the image. A transcoder parameter may also be provided to scale transcoder parameters such as quantization step sizes and motion vectors that are received from the decoder. The scaled transcoder parameters may be fed to an encoder or a transcoder to encode or transcode the scaled images without motion estimation that is computationally expensive.

Claims

1. A method of scaling resolution of an image, comprising:

responsive to receiving first input pixel data for upscaling vertically and horizontally;

storing the first input pixel data in a memory;

sending a block of pixels of the first input pixel data from the memory to a vertical scaler;

generating, at the vertical scaler, first scaled pixel data representing the block of pixels upscaled vertically compared to the block of pixels;

receiving, at a horizontal scaler via a first input of a multiplexer, two or more macroblocks of the first scaled pixel data from the vertical scaler without storing the first scaled pixel data in the memory;

generating second scaled pixel data by filtering pixel data for a plurality of pixels spanning across the two or more macroblocks of the first scaled pixel data at a scaling filter in the horizontal scaler, the second scaled pixel data representing a block of pixels upscaled horizontally compared to the two or more macroblocks of the first scaled pixel data; and

generating a vertically and horizontally upscaled version of the image based on the second scaled pixel data; and

responsive to receiving second input pixel data for horizontal scaling without vertical scaling;

receiving, at the horizontal scaler via a second input of the multiplexer, two or more macroblocks of the second input pixel data;

generating third scaled pixel data by filtering pixel data for a plurality of pixel data spanning across the two or more macroblocks of the second input pixel data, the third scaled pixel data representing a block of pixels scaled horizontally compared to the two or more macroblocks of the second input pixel data; and

generating a horizontally scaled version of the image based on the third scaled pixel data.

2. The method of claim 1, wherein the vertical scaler accesses the memory to generate the first scaled pixel data.

3. The method of claim 1, further comprising:

receiving transcoder parameters from a decoder for decoding a compressed image;

generating scaled transcoder parameters by converting or duplicating the received transcoder parameters; and

sending the scaled transcoder parameters to an encoder for encoding the vertically and horizontally upscaled version of the image or the horizontally scaled version of the image.

4. The method of claim 1, wherein the scaling filter comprises a 17-tap polyphase filter.

5. The method of claim 1, further comprising:

responsive to receiving third input pixel data for downscaling vertically and horizontally;

receiving a block of pixels of the third input pixel data at the horizontal scaler via the multiplexer;

generating, at the horizontal scaler, fourth scaled pixel data representing a block of pixels downscaled horizontally compared to the block of pixels of the third input pixel data;

storing the fourth scaled pixel data in the memory;

sending the fourth scaled pixel data from the memory to the vertical scaler;

generating fifth scaled pixel data at the vertical scaler, the fifth scaled pixel data representing a block of pixels downscaled vertically compared to the fourth scaled pixel data; and

generating a vertically and horizontally downscaled version of the image based on the fifth scaled pixel data.

6. The method of claim 5, wherein the first input pixel data and the fourth scaled pixel data are sent to the memory for storage via another multiplexer.

7. The method of claim 5, further comprising generating the first input pixel data, the second input pixel data and the third input pixel data at a decoder by decoding an input stream.

8. A scaler for scaling an image, the scaler comprising:

a memory and a processor, the processor comprising;

a vertical scaler configured to generate first scaled pixel data representing a block of pixels upscaled vertically compared to first input pixel data for upscaling vertically and horizontally;

a multiplexer having at least a first input and a second input, the first input configured to receive the first scaled pixel data, the second input configured to receive second input pixel data for horizontal scaling without vertical scaling; and

a horizontal scaler coupled to the multiplexer and comprising;

an input buffer configured to receive two or more macroblocks of the first scaled pixel data via the multiplexer without storing the first scaled pixel data in the memory separate from the horizontal scaler and the vertical scaler, the input buffer further configured to receive two or more macroblocks of the second input pixel data via the multiplexer,

a scaling filter configured to generate second scaled pixel data by filtering pixel data for a plurality of pixels spanning across the two or more macroblocks of the first scaled pixel data, the second scaled pixel data representing a block of pixels horizontally upscaled compared to the two or more macroblocks of the first scaled pixel data, the scaling filter further configured to generate third scaled pixel data by filtering pixel data for a plurality of pixels spanning across the two or more macroblocks of the second input pixel data, and

an output buffer configured to store the second scaled pixel data or the third scaled pixel data.

9. The scaler of claim 8, wherein the vertical scaler accesses the memory during generation of the first scaled pixel data.

10. The scaler of claim 8, wherein the scaling filter comprises a 17-tap polyphase filter.

11. The scaler of claim 8, wherein the horizontal scaler is further configured to receive a block of pixels of third input pixel data for downscaling vertically and horizontally and generate fourth scaled pixel data representing a block of pixels downscaled horizontally compared to the block of pixels of the third input pixel data, and wherein the vertical scaler is further configured to generate fifth scaled pixel data representing a block of pixels downscaled vertically compared to the fourth scaled pixel data.

12. The scaler of claim 11, further comprising another multiplexer configured to send the first input pixel data and the fourth scaled pixel data to the memory for storage.

13. The method of claim 11, further comprising a decoder for decoding an input stream into the second input pixel data or the third input pixel data.

14. A transcoder system for converting an image in a first format to a second format, comprising:

a memory and a processor, the processor comprising;

a decoder configured to decode the image in the first format and outputting a sequence of a first decoded macroblocks for upscaling vertically and horizontally or second decoded macroblocks for horizontal scaling without vertical scaling;

a scaler comprising;

a vertical scaler configured to generate first scaled pixel data representing a block of pixels upscaled vertically compared to the first decoded macroblocks;

a multiplexer having at least a first input and a second input, the first input configured to receive the first scaled pixel data, the second input configured to receive the second decoded macroblocks;

a horizontal scaler coupled to the multiplexer and comprising;

an input buffer configured to receive two or more macroblocks of the first scaled pixel data via the multiplexer without storing the first scaled pixel data in the memory separate from the horizontal scaler and the vertical scaler, the input buffer further configured to receive two or more macroblocks of the second decoded macroblocks via the multiplexer,

a scaling filter configured to generate second scaled pixel data by filtering pixel data for a plurality of pixels spanning across the two or more macroblocks of the first scaled pixel data, the second scaled pixel data representing a block of pixels horizontally upscaled compared to the two or more macroblocks of the first scaled pixel data, the scaling filter further configured to generate third scaled pixel data by filtering pixel data for a plurality of pixels spanning across the two or more macroblocks of the second decoded macroblocks, and

an output buffer configured to store the second scaled pixel data or the third scaled pixel data; and

an encoder configured to receive the second scaled pixel data or the third scaled pixel data and encode the second scaled pixel data or the third scaled pixel data into a transcoded image in the second format.

15. The transcoder system of claim 14, wherein the scaler converts or duplicates transcoder parameters received from the decoder, the converted or duplicated transcoder parameters sent to the encoder.

16. The transcoder system of claim 14, wherein the vertical scaler is configured to access the memory to generate the first scaled pixel data.

17. The transcoder system of claim 14, wherein the scaler performs YUV format conversion.

18. The transcoder system of claim 14, wherein the scaling filter comprises a 17-tap polyphase filter.

19. The transcoder system of claim 14, wherein the scaler is configured to send the second scaled pixel data or the third scaled pixel data according to the sequence as encoded by the encoder.