System and method for the scaling of image streams that use motion vectors

• US 6,075,906 A
• Filed: 12/13/1995
• Issued: 06/13/2000
• Est. Priority Date: 12/13/1995
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Silicon Graphics Incorporated (Graphics Properties Holdings, Inc.)
• Inventors: Fenwick, Stephen C., Van Hook, Timothy J., Efland, Gregory Humpreys
• Primary Examiner(s): Tran, Phuoc
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US08/572,299
• Time in Patent Office: 1,644 Days
• Field of Search: 382/236, 382/248, 382/250, 382/251, 382/252, 382/253, 382/298, 382/232, 348/402, 348/413, 348/411, 348/581, 358/451
• US Class Current: 382/298
• CPC Class Codes: G06T 3/4023 based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/428 Recompression, e.g. by spatial or temporal decimation
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen

Abstract

A system and method for scaling image streams that use motion vectors is disclosed. The system combines an error term with a predicted term in order to produce a display value. The system operates on image components represented in the spatial and frequency domains. The system processes motion vectors in the spatial domain. The motion vectors are scaled. The integral part of the scaled motion vector addresses a framestore. The fractional portion of the scaled motion vector is input to a nonlinear filter which determines the value of image components for a location that does not correspond with an image location in the framestore. The output of the nonlinear filter comprises the predicted terms. Data in the frequency domain is processed more efficiently by reducing the size of a block of data by appropriate filtering. The resulting data is transformed to the spatial domain.

Claims

1. A method for scaling an image contained in an image stream, wherein the image stream comprises a motion vector used to scale the image, and a block of error terms to scale the image, comprising the steps of:

(1) separating a block of data corresponding to a frequency domain representation of the block of error terms from the image stream;

(2) low pass filtering said frequency domain representation according to a scale factor to produce a reduced-size low pass filtered block;

(3) inverse quantizing said reduced-size low pass filtered block to produce a reduced-size inverse quantized block;

(4) selectively transforming said reduced-size inverse quantized block to a spatial domain representation according to said scale factor to produce a decimated representation of the block of error terms;

(5) separating the motion vector from the image stream;

(6) scaling the motion vector according to a scale factor, thereby producing a scaled motion vector;

(7) addressing an image framestore with said scaled motion vector to obtain a block of image components corresponding to said scaled motion vector;

(8) nonlinear filtering said block of said image components with a nonlinear filter, thereby producing a block of predicted terms; and

(9) using one of said block of error terms, said block of predicted terms, or a combination of said block of error terms and said block of predicted terms to form a block of display pixels.

2. The method of claim 1, wherein steps (1)-(4) occur substantially simultaneously with steps (5)-(8).

3. The method of claim 1, wherein steps (1)-(4) occur asynchronously with steps (5)-(8).

4. A method for scaling an image contained in an image stream according to a scale factor, wherein the image stream comprises a motion vector, comprising the steps of:

(1) separating from the image stream a block of data corresponding to a frequency domain representation of a block of error terms;

(2) low pass filtering said frequency domain representation according to the scale factor to produce a reduced-size low pass filtered block;

(3) inverse quantizing said reduced-size low pass filtered block to produce a reduced size inverse quantized block;

(4) selectively transforming said reduced-size inverse quantized block to a spatial domain representation according to the scale factor to produce a decimated representation of said block of error terms;

(5) separating the motion vector from the image stream;

(6) scaling the motion vector according to the scale factor to produce a scaled motion vector;

(7) outputting from a framestore a block of image components identified by an integer component of said scaled motion vector;

(8) filtering said block of image components outputted from said framestore using a nonlinear filter, thereby producing a block of predicted terms; and

(9) using one of said block of error terms, said block of predicted terms, or a combination of said block of error terms and said block of predicted terms to form a block of display pixels.

5. The method of claim 4, wherein said filtering step further comprises the step of using a fractional component of said scaled motion vector to determine a set of filter coefficients.

6. The method of claim 5, wherein said fractional component of said scaled motion vector is used as an index into a look-up table comprising a plurality of filter coefficients.

7. The method of claim 4, wherein steps (1)-(4) occur substantially simultaneously with steps (5)-(8).

8. The method of claim 4, wherein steps (1)-(4) occur asynchronously with steps (5)-(8).

9. A system for scaling an image contained in an image stream, wherein said image stream comprises a motion vector and a block of error terms, comprising:

a framestore having image components stored therein;

a scale adjustment module to scale the motion vector according to a scale factor to produce a scaled motion vector, wherein an integer component of said scaled motion vector is provided to said framestore, and said framestore outputs a block of image components identified by said integer component of said scaled motion vector;

a first filter to filter said block of image components outputted from said framestore and to output a block of predicted terms, wherein said first filter is a nonlinear filter;

a second filter to low pass filter a frequency domain representation of the block of error terms according to said scale factor to produce a reduced-size low pass filtered block of error terms;

an IQTZ module to inverse quantize said reduced-size low pass filtered block of error terms to produce a reduced-size inverse quantized block of error terms;

an IDCT module to transform said reduced-size inverse quantized block of error terms to a spatial domain representation according to said scale factor to produce a decimated representation of the block of error terms.

10. The system of claim 9, wherein a fractional component of said scaled motion vector is provided to said first filter, wherein said first filter uses said fractional component in filtering said block of image components outputted from said framestore.

11. The system of claim 10, wherein said first filter determines a set of filter coefficients based on said fractional component of said scaled motion vector.

12. The system of claim 11, wherein said first filter comprises a look-up table comprising a plurality of sets of filter coefficients, wherein said fractional component of said scaled motion vector is used as an index into said look-up table.

13. The system of claim 11, wherein said first filter comprises a computation unit to determine said set of filter coefficients.

14. The system of claim 9, further comprising:

a summer to add each predicted term in said block of predicted terms with a corresponding error term to produce a block of display image components;

a display framestore to store said block of display image components; and

means for displaying said block of display image components.