Method and apparatus for performing two pass quality video compression through pipelining and buffer management

• US 6,760,478 B1
• Filed: 07/10/2000
• Issued: 07/06/2004
• Est. Priority Date: 06/21/1996
• Status: Expired due to Fees

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Galbi, Duane E., Adiletta, Matthew James, Ho, Samuel William, Wheeler, William Ralph, Rose, Robert Clint, Chow, King-Wai
• Primary Examiner(s): Chen, Wenpeng
• Application Number:
  US09/613,359
• Time in Patent Office: 1,457 Days
• Field of Search: 382/236, 382/232, 382/234, 382/239, 348/416, 348/699, 348/402, 348/409, 348/390, 348/415, 348/413, 348/405, 348/384, 375/340.02, 375/240.01, 375/240.12, 375/240.13, 375/240.16, 375/240.14
• US Class Current: 382/236
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 19/93 : Run-length coding

Abstract

An apparatus and method for performing two-pass real time video compression is provided. Tactical decisions such as encoding and quantization values are determined in software, whereas functional execution steps are performed in hardware. By appropriately apportioning the tasks between software and hardware, the benefits of each type of processing are exploited, while minimizing both hardware complexity and data transfer requirements. One key concept that allows the compression unit to operate in real time is that the architecture and pipelining both allow for B frames to be executed out of order. By buffering B frames, two-pass motion estimation techniques can be performed to tailor bit usage to the requirements of the frame, and therefore provide a more appealing output image.

Claims

1. A method of providing compression of video data input to a video compression/decompression unit (VCDU) coupled to a central processor unit (CPU), said video data comprising a plurality of frames, including predictive frames and bi-directional frames, each of said frames apportioned into a plurality of macroblocks, each of said macroblocks comprising a plurality of pixels, said method comprising the steps of:

said VCDU comparing each of said frames to other ones of said frames to estimate the motion characteristics of each of said macroblocks of each of said frames;

determining, by said CPU, a coding type for each of said macroblocks of each of said frames responsive to said motion characteristics provided from said VCDU;

said VCDU encoding each of said macroblocks of each of said frames responsive to said coding type determined by said CPU to provide one or more tokens for each of said plurality of frames;

buffering, by said CPU, selected ones of said bi-directional frames and processing the buffered frames out of order in which said plurality of frames initially occur;

translating, by said CPU, said one or more tokens for each of said plurality of frames into a bit stream for storage in a storage device; and

wherein said VCDU and said CPU are connected by an input-output bus and the steps of the VCDU comparing and the CPU determining provide for minimal traffic on the input-output bus while tasks are efficiently apportioned between said VCDU and said CPU.

2. The method according to claim 1, wherein said step of said VCDU comparing each of said frames further comprises the steps of:

determining at least one motion vector for each of said macroblocks, said at least one motion vector indicating a motion characteristic between said each macroblock and a similar macroblock in a preceding or succeeding frame.

3. The method according to claim 1, wherein said step of said VCDU comparing each of said frames further comprises the steps of determining energy characteristics of each of said macroblocks.

4. The method according to claim 1, wherein said step of said VCDU comparing each of said frames further comprises the steps of:

determining a first and second motion vector for each of said macroblocks, said first and second motion vectors indicating a motion characteristic between said each macroblock and a similar macroblock in a preceding and succeeding frame, respectively; and

determining energy characteristics of each of said macroblocks.

5. The method according to claim 2 wherein said step of said determining, by said CPU, a coding type for each of said macroblocks of each of said frames responsive to said motion characteristics further comprises the steps of:

designating each of said macroblocks as either spatial or temporal coding type; and

assigning a quantization value to each of said macroblocks, said quantization value for controlling the number of bits that are to be used to encode the macroblock.

6. The method according to claim 4 wherein said step of determining, by said CPU, a coding type for each of said macroblocks of each of said frames responsive to said motion characteristics further comprises the steps of:

designating each of said macroblocks as either spatial or temporal coding type; and

assigning a quantization value to each of said macroblocks, said quantization value for controlling the number of bits that are to be used to encode the macroblock.

7. The method according to claim 6, wherein said quantization value is assigned to each of said macroblocks responsive to the energy characteristics of other macroblocks within said associated frame.

8. The method according to claim 6, wherein the step of said VCDU encoding each of said macroblocks of each of said frames responsive to said coding type comprises the steps of:

responsive to said coding type being a temporal coding type and said macroblock being a reference type macroblock, providing an error macroblock by calculating the difference between the value of pixels within said macroblock and the value of pixels within the macroblock indicated by said first motion vector;

responsive to said coding type being a temporal coding type and said macroblock being a non-reference type macroblock, providing an error macroblock by calculating the difference between the value of pixels within said macroblock and the average of value of pixels within the macroblock indicated by said first motion vector and the value of pixels within the macroblock indicated by said second motion vector.

9. The method according to claim 8, wherein the step of said VCDU encoding each of said macroblocks further comprises the step of:

applying a discrete cosine transform to either said macroblock, or said error macroblock, responsive to said coding type being either a spatial coding type or a temporal coding type, respectively, to provide a transformed macroblock.

10. The method according to claim 9, wherein the step of said VCDU encoding each of said macroblocks further comprises the step of:

quantizing each of the macroblocks by multiplying the quantization value provided from said CPU to each of the entries of said transformed macroblock to provide a quantized macroblock.

11. The method according to claim 10, further comprising the step of:

generating one or more tokens for each of said macroblocks, said tokens generated by traversing said quantized macroblock and recording the distance between frequency components in said quantized macroblock.

12. The method according to claim 1, wherein the step of translating, by said CPU, said one or more tokens for each of said plurality of frames into a bit stream for storage in a storage device includes the steps of defining, for each unique one of said one or more tokens, a compact code.

13. A method of performing decompression of a bit stream representing video data stored in a storage device of a computer system, said computer system including a central processing unit (CPU) and a video compression/decompression unit (VCDU) coupled to said CPU, said video data comprising a plurality of frames, each of said frames apportioned into a plurality of macroblocks, said method comprising the steps of:

translating, by said CPU, said bit stream into one or more tokens for each of said plurality of frames;

said VCDU decoding each of said tokens to construct a plurality of macroblocks for each of said frames;

decoding, by said CPU, a coding type for each of said macroblocks of each of said frames, said coding type for indicating how each of said macroblocks is compressed; and

using at least one stored motion vector for each of said macroblocks, and responsive to said coding type of each of said macroblocks, reconstructing each macroblock to provide a restored video image;

wherein said coding types include temporal coding types and spatial coding types, and wherein said step of reconstructing each said macroblock further comprises the step of;

responsive to said coding type being a temporal coding type providing said restored macroblock by adding the contents of each entry of said macroblock to the value of pixels within the macroblock indicated by a first motion vector;

applying an inverse discrete cosine transform to either said constructed macroblock, or said restored macroblock, responsive to said coding type being either a spatial coding type or a temporal coding type, respectively, to provide a reconstructed macroblock for use in said restored video image;

wherein the macroblock is double-buffered between a quantization process and a discrete cosine transform process; and

wherein said VCDU and said CPU are connected by an input-output bus and in a compression process the VCDU compares frames to estimate motion characteristics of macroblocks and the CPU determines code types of the macroblocks based on the motion characteristics from the VCDU, said comparing and determining providing minimal traffic on the input-output bus while tasks are efficiently apportioned between said VCDU and said CPU.

14. An apparatus for performing compression and decompression of video data input to a video compression/decompression unit (VCDU) coupled via interface means to a central processor unit (CPU), said video data comprising a plurality of frames, including predictive frames and bi-directional frames, each of said frames apportioned into a plurality of macroblocks, each of said macroblocks comprising a plurality of pixels, said VCDU further comprising:

motion estimation means, in said VCDU for comparing each of said frames to other ones of said frames to estimate the motion characteristics of each of said macroblocks of each of said frames;

means for encoding each of said macroblocks of each of said frames responsive to a coding type determined by said CPU to provide one or more tokens for each of said plurality of frames;

means for buffering, by said CPU, selected ones of said bi-directional frames and for processing the buffered frames out of order in which said plurality of frames initially occur; and

wherein said interface means comprise an input-output bus and the motion estimation means and means for encoding provide minimal traffic on the input-output bus while each means efficiently apportions tasks between said VCDU and said CPU.

15. The apparatus of claim 14, wherein said CPU further comprises:

means for determining said coding type for each of said macroblocks of each of said frames responsive to said motion characteristics provided from said VCDU via said interface means; and

translating means for converting said one or more tokens received via said interface means from said VCDU for each of said plurality of frames into a bit stream for storage in a storage device.

16. The apparatus of claim 14 further comprising:

reconstruction means, in said VCDU for translating said bit stream representing said plurality of frames into frame data.

17. The apparatus of claim 16 wherein said motion estimation means is coupled to receive video data from an external device and is further coupled to receive frame data from said reconstruction means.