Decoder for a software-implemented end-to-end scalable video delivery system

• US 5,742,892 A
• Filed: 04/18/1995
• Issued: 04/21/1998
• Est. Priority Date: 04/18/1995
• Status: Expired due to Term

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Chaddha, Navin
• Primary Examiner(s): Peng, John K.
• Assistant Examiner(s): Srivastava, Vivek
• Application Number:
  US08/424,703
• Time in Patent Office: 1,099 Days
• Field of Search: 348/400, 348/398, 348/416, 348/417, 348/422, 348/405, 348/412, 455/5.1, 455/3.1
• US Class Current: 725/146
• CPC Class Codes: G06T 9/008 Vector quantisation
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/94 Vector quantisation
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00

Abstract

A decoder is used in an end-to-end scalable video delivery system operable over heterogeneous networks. The decoder may software-based and computationally low complexity, or may be implemented inexpensively in ROM hardware. The system utilizes a scalable video compression algorithm based on a Laplacian pyramid decomposition to generate an embedded information stream. At the receiving end, the decoder extracts from the embedded stream different streams at different spatial and temporal resolutions. Decoding a 160×120 pixel image involves only decompressing a base layer 160×120 pixel image. Decoding a 320×240 pixel image involves decompressing and up-sampling (e.g., interpolating) the base layer to yield a 320×240 pixel image to which is added error data in a first enhancement layer following its decompression. To obtain a 640×480 pixel image, the decoder up-samples the up-sampled 320×240 pixel image, to which is added error data in a second enhancement layer, following its decompression. Because decoding requires only additions and look-ups from a table stored in a small (12 Kb) memory, decoding occurs in real-time. Subjective quality of the compressed images preferably is enhanced using perceptual distortion measures. The system also provides joint-source channel coding capability on heterogenous networks. The look-up table or codebook includes the inverse perceptual weighting (preprocessed) and the inverse transform (preprocessed). Decoding permits the codewords within the look-up table codebook to include preprocessed color conversion, dithering, color palletization, edge-enhancement, decimation, and interpolation.

Claims

1. A decoder for use with a video delivery system whose server provides an embedded bit stream in pixel blocks including information for at least two spatial resolutions and transmittable over at least one network, the decoder including:

a central processor unit coupled to a memory unit;

a look-up table, stored in said memory unit, including preprocessed decoded versions of quantized indexable representations of inverse discrete cosine transforms of image data used by said server in creating codewords present in said embedded bit stream; and

means, operating under control of said central processor unit, for processing said information contained in said embedded bit stream;

said means for processing decoding a first spatial resolution image by decompressing base layer data contained within said embedded bit stream;

said means for processing further decoding a second, higher, spatial resolution image by decompressing said base layer data to obtain a first intermediate image that is up-sampled to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream;

wherein said server encodes said embedded bit stream by performing a discrete cosine transform followed by tree-structured quantization, and

wherein said decoder can decode a viewable image from data in said embedded bit stream from received codewords containing less than a number of bits representing a full-length codeword.

2. The decoder of claim 1, wherein said embedded bit stream contains at least three spatial resolutions, and wherein said means for processing decodes a third image whose spatial resolution is higher than said second image by up-sampling said first up-sampled image to yield a second intermediate image to which is added decompressed error data in a second enhancement layer contained in said embedded bit stream.

3. The decoder of claim 1, wherein said memory unit stores at least one of the following (i) an algorithm comprising said means for processing, (ii) look-up data used in decompressing said base layer data, (iii) look-up data associated with said error data in said first enhancement layer, (iv) preprocessed data representing up-sampling of said first up-sampled image, and (v) preprocessed data associated with said error data in said second enhancement layer.

6. The decoder of claim 1, wherein said server encodes spatial resolution data in said embedded bit stream in pixel blocks, wherein:

said decoder provides error correction for said embedded bit stream using inverse vector quantization followed by inverse discrete cosine transformation of at least some of said blocks of data, said inverse vector quantization and said inverse discrete cosine transformation being preprocessed and stored in said memory unit.

7. The decoder of claim 1, wherein said vector quantization includes human perception modelling.

8. The decoder of claim 1, wherein said memory unit stores codewords arranged in a tree structure.

9. The decoder of claim 1, wherein said tree-structured vector quantization has a tree depth R and has a vector dimension is k, and wherein bitstream bit rates O/k, . . . , R/k are provided for said embedded bit stream.

10. The decoder of claim 2, wherein:

said first image has 160×

120 pixel resolution, said second image has 320×

240 pixel resolution, and wherein said third image has 640×

480 pixel resolution;

said means for processing decodes a 160×

120 pixel image by decompressing base layer data contained within said embedded bit stream;

said means for processing decodes a 320×

240 pixel image by decompressing said base layer data to obtain a first intermediate image and up-sampling said first intermediate image to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream; and

said means for processing decodes a 640×

480 pixel image by up-sampling said first up-sampled image to yield a second intermediate image to which is added decompressed error data in a second enhancement layer contained in said embedded bit stream.

4. A decoder for use with a video delivery system whose server provides an embedded bit stream, transmittable over at least one network, including information for at least two spatial resolutions wherein for at least one of said two spatial resolutions domain information provided has been vector quantized, the decoder including:

a central processor unit coupled to a memory unit;

a look-up table, stored in said memory unit, including preprocessed decoded versions of quantized indexable representations of frequency domain transforms of image data used by said server in creating codewords present in said embedded bit stream;

means, operating under control of said central processor unit, for processing said information contained in said embedded bit stream;

said means for processing decoding a first spatial resolution image by decompressing base layer data contained within said embedded bit stream;

said means for processing further decoding a second, higher, spatial resolution image by decompressing said base layer data to obtain a first intermediate image that is up-sampled to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream;

wherein said server encodes said embedded bit stream by performing a discrete cosine transform followed by tree-structured vector quantization, and

wherein vector quantization is such that said means for processing can decode a viewable image from data in said embedded bit stream from received codewords containing less than a number of bits representing a full-length codeword.

5. The decoder of claim 4, wherein said tree-structured vector quantization has a tree depth R and a vector dimension k;

wherein bitstream bit rates O/k, . . . , R/k are provided for said embedded bit stream; and

wherein said memory unit stores codewords arranged in a tree structure.

11. The decoder of claim 4, wherein said vector quantization includes human perception modelling.

12. A hardware-based decoder for use with a video delivery system whose server provides an embedded bit stream in pixel blocks including information for at least two spatial resolutions and transmittable over at least one network, the decoder including:

a central processor unit coupled to a memory unit including a read-only memory;

said read-only memory including preprocessed decoded versions of quantized indexable representations of inverse discrete cosine transforms of image data used by said server in creating codewords present in said embedded bit stream;

said preprocessed decoded versions including at least information used to decode a first spatial resolution image by decompressing base layer data contained within said embedded bit stream; and

information used to decode a second, higher, spatial resolution image by decompressing said base layer data to obtain a first intermediate image that is up-sampled to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream;

wherein said server encodes said embedded bit stream by performing a discrete cosine transform followed by tree-structured quantization, and

wherein said decoder can decode a viewable image from data in said embedded bit stream from received codewords containing less than a number of bits representing a full-length codeword.

13. The decoder of claim 12, wherein said embedded bit stream contains at least three spatial resolutions, and wherein said preprocessed decoded versions further include informed used to decode a third spatial resolution image, whose spatial resolution is higher than said second image, by up-sampling said first up-sampled image to yield a second intermediate image to which is added decompressed error data in a second enhancement layer contained in said embedded bit stream.

14. The decoder of claim 12,

wherein said read-only memory stores codewords arranged in a tree structure.

15. The decoder of claim 12, wherein said vector quantization includes human perception modelling.

16. A method of decoding information received by a decoder used with a video delivery system whose server provides an embedded bit stream in pixel blocks including information for at least two spatial resolutions and transmittable over at least one network, the method including the following steps:

(a) providing a stored set of preprocessed decoded versions of quantized indexable representations of inverse discrete cosine transforms of image data used by said server in creating codewords present in said embedded bit stream;

(b) processing said information in said embedded bit stream, including using information contained in said bit stream to index into said stored set of preprocessed decoded versions to decode a first spatial resolution image by decompressing base layer data contained within said embedded bit stream and to decode a second, higher, spatial resolution image by decompressing said base layer data to obtain a first intermediate image and up-sampling said first intermediate image to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream;

wherein said server encodes said embedded bit stream by performing a discrete cosine transform followed by tree-structured quantization, and

wherein said decoder can decode a viewable image from data in said embedded bit stream from received codewords containing less than a number of bits representing a full-length codeword.

17. The method of claim 16, wherein said embedded bit stream contains at least three spatial resolutions, and wherein:

step (b) includes decoding a third image whose spatial resolution is higher than said second image by up-sampling said first up-sampled image to yield a second intermediate image to which is added decompressed error data in a second enhancement layer contained in said embedded bit stream.

18. The method of claim 16, wherein said tree-structured vector quantization has a tree depth R, a vector dimension k, and wherein bitstream bit rates O/k, . . . , R/k are provided for said embedded bit stream.