Coded domain picture composition for multimedia communications systems

• US 5,629,736 A
• Filed: 11/01/1994
• Issued: 05/13/1997
• Est. Priority Date: 11/01/1994
• Status: Expired due to Term

• Current Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Yan, Li, Haskell, Barin G.
• Primary Examiner(s): Lee, Michael
• Application Number:
  US08/332,985
• Time in Patent Office: 924 Days
• Field of Search: 348/563-568, 348/554, 348/555, 348/556, 348/584, 348/588, 348/385, 348/388, 348/387, 348/370, 348/393, 348/394, 348/398, 348/403, 348/404, 348/405, 348/407
• US Class Current: 348/386.1
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

Video composition techniques are disclosed for processing video information from a plurality of sources to provide a video image having a plurality of rectangular regions. Each rectangular region displays video information from a specific one of the plurality of video sources. The video information from each video source is in the form of an incoming digital bit stream. The digital bit stream from a first video source has a first bit rate, and the digital bit stream from a second video source has a second bit rate where the first bit rate may or may not be equal to the second bit rate. The incoming digital bit streams are fed to a rate matching circuit which converts all incoming digital bit streams to a common bit rate. The output of the rate matching circuit is fed to a synchronization arid multiplexer circuit which places video information from specific digital bit streams into corresponding rectangular regions of a composite video image.

Claims

1. A compressed domain video composition system for integrating up to four coded video input bit streams into a composite video output bit stream in a compressed domain, the system comprising:

(a) a bit rate reduction device for selectively applying bit rate reduction to the up to four coded video input bit streams to generate a set of bit rate reduced video bit streams each having a corresponding rate-reduced bit rate,

(b) a compositing device for combining the bit rate reduced video bit streams into the composite video output bit stream, the composite video output bit stream having a bit transfer rate equal to the sum of the rate-reduced bit rates,

wherein the bit rate reduction device selectively applies rate reduction to any of the up to four coded video input bit streams so as to provide a composite video output bit stream having a desired bit transfer rate;

(c) up to four buffers for receiving a corresponding plurality of up to four coded video input bit streams;

(d) a synchronization and multiplexing device for synchronizing the up to four coded video input bit streams and multiplexing the up to four coded video input bit streams into a single, compressed, composite video output bit stream;

(e) for each of the up to four coded video input bit streams, a corresponding transmission rate reduction system wherein a first compressed video bit stream having a first transfer rate is converted into a second compressed video bit stream having a second transfer rate, the second transfer rate being slower than the first transfer rate, the transmission rate reduction system comprising;

(i) a video multiplex decoder for demultiplexing and decoding the first video bit stream to generate (a) quantized DCT coefficient microblocks representing a plurality of digitized frequency components for the first video bit stream, (b) motion vector pairs specifying a vertical displacement and a horizontal displacement for each microblock; and

(c) header information specifying a starting point for a compressed video image, compressed video image synchronization information, and the location of each microblock or group of microblocks in the compressed video image;

(ii) a DCT coefficients processor for receiving quantized DCT coefficient microblocks from the video multiplex decoder and receiving quantization information from a rate control processor to generate a set of processed DCT coefficient microblocks containing fewer DCT coefficients than the number of coefficients in the quantized DCT coefficient microblocks received from the video multiplex decoder; and

(iii) a video multiplex encoder for encoding requantized DCT coefficient microblocks obtained from the DCT coefficients processor, for encoding a motion vector obtained from the DCT coefficients processor, and for multiplexing header information with the encoded motion vector and encoded DCT coefficient microblocks to produce the second compressed video bit stream; and

(f) a central rate control processor providing control information for the up to four DCT coefficients processors, the rate control processor comprising a processor for generating new quantization information for the up to four DCT coefficients processors, each DCT coefficients processor coupled to a transmission buffer, the new quantization information generated according to a transmission buffer status value specifying the number of available bits remaining in the video transmission buffer, and according to a desired bit rate for the composite video output bit stream.

2. A compressed domain video composition system as set forth in claim 1 wherein each of the up to four DCT coefficients processors further includes:

(a) a first inverse quantizer which, in response to the receipt of the quantized DCT coefficients and the quantization information from the video multiplex decoder, produces an output in the form of reconstructed DCT coefficients according to the quantization information;

(b) a quantizer which accepts the reconstructed DCT coefficients and a feedback DCT error signal to produce requantized DCT coefficients according to the new quantization information obtained from the rate control processor;

(c) a second inverse quantizer which receives the requantized DCT coefficients and produces reconstructed DCT coefficients based upon the new quantization information;

(d) a prediction frame storage device which stores a frame including a plurality of DCT coefficient blocks;

(e) a motion vector processor which receives a plurality of motion vector pairs from the video multiplex decoder and processes the motion vector pairs such that respective processed motion vector pairs are aligned with corresponding DCT coefficient blocks;

(f) a first summing device coupled to the output of the first inverse quantizer, the output of the frame storage device, and the input of the quantizer, for adding the output of the first inverse quantizer and the feedback DCT error signal from the output of the frame storage device to generate a sum if a coding mode of intercoding is used that incorporates a reference to a previously occurring frame, and applying the sum so generated to the input of the quantizer;

(g) a second summing device coupled to the first inverse quantizer and the second inverse quantizer for subtracting the output of the second inverse quantizer from the output of the first inverse quantizer to produce the DCT error signal and for applying the DCT error signal to the input of a third summing device;

(h) a third summing device coupled to the second summing device and the frame storage device for performing motion compensation for the DCT error signal.

3. A central rate control processor for use in a compressed domain video composition system including:

(i) an input buffer for storing information from a plurality of coded video input bit streams, each coded video input bit stream having a corresponding input bit transfer rate, (ii) a bit rate reducing device coupled to the set of buffers for selectively reducing the bit transfer rates of any of the coded video input bit streams, (iii) a compositing device for combining the bit rate reduced video bit streams into the composite video output bit stream, (iv) a plurality of DCT coefficients processors, each DCT coefficients processor coupled to a corresponding coded video input bit stream, and (v) a multiple transmission buffer including a plurality of memory locations and coupled to the compositing device;

CHARACTERIZED IN THAT

the central rate control processor includes;

(a) a control signal generator for generating multiple control signals causing the selective application of bit rate reduction to the plurality of coded video input bit streams to generate a set of bit rate reduced video bit streams each having a corresponding rate-reduced bit rate, so as to provide a composite video output bit stream having a desired bit transfer rate, and wherein the desired bit transfer rate is substantially equal to the sum of the bit rate reduced video bit streams, and

(b) a monitoring device coupled to the multiple transmission buffer for monitoring the relative number of loaded memory locations in the multiple transmission buffer, the relative number of loaded memory locations denoted as the transmission buffer status value;

wherein the central rate control processor provides control information for the plurality of DCT coefficients processors, the rate control processor generating new quantization information for the plurality of DCT coefficients processors, each DCT coefficients processor coupled to the transmission buffer, the new quantization information generated according to the transmission buffer status value specifying the number of available bits remaining in the transmission buffer, and according to a desired bit rate for the composite video output bit stream.