Decoder for a software-implemented end-to-end scalable video delivery system

• US 6,266,817 B1
• Filed: 04/20/1998
• Issued: 07/24/2001
• Est. Priority Date: 04/18/1995
• Status: Expired due to Term

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Chaddha, Navid
• Primary Examiner(s): Faile, Andrew
• Assistant Examiner(s): Srivastava, Vivek
• Application Number:
  US09/062,879
• Time in Patent Office: 1,191 Days
• Field of Search: 345/327, 458/5.1, 348/400, 348/398, 348/416, 348/417, 348/422, 348/413.01, 348/402.01, 348/403.01, 375/240.11, 375/240.22, 375/240.03, 375/240.2
• US Class Current: 725/146
• CPC Class Codes: G06T 9/008 Vector quantisation
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/94 Vector quantisation
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00

Abstract

A decoder is used in an end-to-end scalable video delivery system operable over heterogeneous networks. The decoder may software-based and computationally low complexity, or may be implemented inexpensively in ROM hardware. The system utilizes a scalable video compression algorithm based on a Laplacian pyramid decomposition to generate an embedded information stream. At the receiving end, the decoder extracts from the embedded stream different streams at different spatial and temporal resolutions. Decoding a 160×120 pixel image involves only decompressing a base layer 160×120 pixel image. Decoding a 320×240 pixel image involves decompressing and up-sampling (e.g., interpolating) the base layer to yield a 320×240 pixel image to which is added error data in a first enhancement layer following its decompression. To obtain a 640×480 pixel image, the decoder up-samples the up-sampled 320×240 pixel image, to which is added error data in a second enhancement layer, following its decompression. Because decoding requires only additions and look-ups from a table stored in a small (12 Kb) memory, decoding occurs in real-time. Subjective quality of the compressed images preferably is enhanced using perceptual distortion measures. The system also provides joint-source channel coding capability on heterogenous networks. The look-up table or codebook includes the inverse perceptual weighting (preprocessed) and the inverse transform (preprocessed). Decoding permits the codewords within the look-up table codebook to include preprocessed color conversion, dithering, color palletization, edge-enhancement, decimation, and interpolation.

Claims

1. A decoder that receives a network-transmittable server generated single embedded bit stream in pixel blocks that contains information for at least two spatial resolutions of video, said server encoding said embedded bit stream using discrete cosine transformation followed by tree-structured quantization, said decoder outputting from said bit stream scaleable video that is variable in at least one of spatial resolution, temporal resolution, and data rate, the decoder including:

a central processor unit coupled to a memory unit;

a look-up table, stored in said memory unit, including preprocessed decoded versions of quantized indexable representations of inverse discrete cosine transforms of image data used when codewords present in said embedded bit stream were created; and

a processor controlled by said central processor unit that processes said information contained in said embedded bit stream and decodes at least a first spatial resolution image by decompressing base layer data contained within said embedded bit stream;

wherein said decoder can decode a viewable image from data in said embedded bit stream from received codewords containing less than a number of bits representing a full-length codeword.

2. The decoder of claim 1, wherein said processor further decodes a second, higher, spatial resolution image by decompressing said base layer data to obtain a first intermediate image that is up-sampled to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream.

3. The decoder of claim 2, wherein said embedded bit stream contains at least three spatial resolutions, and wherein said processor further decodes a third image whose spatial resolution is higher than said second image by up-sampling said first up-sampled image to yield a second intermediate image to which is added decompressed error data in a second enhancement layer contained in said embedded bit stream.

4. The decoder of claim 2, wherein said memory unit stores at least one of the following (i) an algorithm used to carry-out said decoding, (ii) look-up data used in decompressing said base layer data, (iii) look-up data associated with said error data in said first enhancement layer, (iv) preprocessed data representing up-sampling of said first up-sampled image, and (v) preprocessed data associated with said error data in said second enhancement layer.

5. The decoder of claim 2, wherein said tree-structured vector quantization has a tree depth R and a vector dimension k;

wherein bitstream bit rates O/k . . . , R/k are provided for said embedded bit stream.

6. The decoder of claim 5, wherein said memory unit stores codewords arranged in a tree structure.

7. The decoder of claim 1, wherein said server encodes spatial resolution data in said embedded bit stream in pixel blocks, wherein:

said decoder provides error correction for said embedded bit stream using inverse vector quantization followed by inverse discrete cosine transformation of at least some of said pixel blocks, said inverse vector quantization and said inverse discrete cosine transformation being preprocessed and stored in said memory unit.

8. The decoder of claim 1, wherein said vector quantization includes human perception modelling.

9. The decoder of claim 1, wherein said memory unit stores codewords arranged in a tree structure.

10. The decoder of claim 2, wherein:

said first image has 160×

120 pixel resolution, said second image has 320×

240 pixel resolution, and wherein said third image has 640×

480 pixel resolution.

11. The decoder of claim 2, wherein:

said processor decodes a 160×

120 pixel image by decompressing base layer data contained within said embedded bit stream;

said processor decodes a 320×

240 pixel image by decompressing said base layer data to obtain a first intermediate image and up-samples said first intermediate image to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream; and

said processor decodes a 640×

480 pixel image by up-sampling said first up-sampled image to yield a second intermediate image to which is added decompressed error data in a second enhancement layer contained in said embedded bit stream.

13. The decoder of claim 1 wherein said preprocessed decoded versions further include information used to decode a second, higher, spatial resolution image by decompressing said base layer data to obtain a first intermediate image that is up-sampled to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream.

12. A hardware-based decoder for use with a video delivery system whose server encodes an embedded bit stream in pixel blocks using discrete cosine transformation followed by tree-structured quantization, said embedded bit stream including information for at least two spatial resolutions and transmittable over at least one network, the decoder including:

a central processor unit coupled to a memory unit including a read-only memory storing preprocessed decoded versions of quantized indexable representations of inverse discrete cosine transforms of image data used by said server to create codewords present in said embedded bit stream;

said preprocessed decoded versions including at least information used to decode a first spatial resolution image by decompressing base layer data contained within said embedded bit stream;

wherein said decoder can decode a viewable image from data in said embedded bit stream from received codewords containing less than a number of bits representing a full-length codeword.

14. The decoder of claim 12, wherein said embedded bit stream contains at least three spatial resolutions, and wherein said preprocessed decoded versions further include information used to decode a third spatial resolution image, whose spatial resolution is higher than said second image, by up-sampling said first up-sampled image to yield a second intermediate image to which is added decompressed error data in a second enhancement layer contained in said embedded bit stream.

15. The decoder of claim 14, wherein said memory unit stores at least one of the following (i) an algorithm used to carry-out said decoding, (ii) look-up data used in decompressing said base layer data, (iii) look-up data associated with said error data in said first enhancement layer, (iv) preprocessed data representing up-sampling of said first up-sampled image, and (v) preprocessed data associated with said error data in said second enhancement layer.

16. The decoder of claim 12, wherein said tree-structured vector quantization has a tree depth R and a vector dimension k;

wherein bitstream bit rates O/k . . . , R/k are provided for said embedded bit stream.

17. The decoder of claim 12, wherein said read-only memory stores codewords arranged in a tree structure.

18. The decoder of claim 12, wherein said vector quantization includes human perception modelling.

19. The decoder of claim 12, wherein:

said server encodes spatial resolution data in said embedded bit stream in pixel blocks; and

wherein;

said decoder provides error correction for said embedded bit stream using inverse vector quantization followed by inverse discrete cosine transformation of at least some of said pixel blocks, said inverse vector quantization and said inverse discrete cosine transformation being preprocessed and stored in said memory unit.

20. A method of decoding information embedded by a video server in an embedded bit stream having pixel blocks that include information for at least two spatial resolutions, said embedded bit stream being encoded by said server using discrete cosine transformation followed by tree-structured quantization, the method including the following steps:

(a) providing a stored set of preprocessed decoded versions of quantized indexable representations of inverse discrete cosine transforms of image data used by said server in creating codewords present in said embedded bit stream;

(b) processing said embedded bit stream to use information contained therein to index into said stored set of preprocessed decoded versions to decode at least one of;

(i) a first spatial resolution image by decompressing base layer data contained within said embedded bit stream;

(ii) a second, higher, spatial resolution image by decompressing said base layer data to obtain a first intermediate image and up-sampling said first intermediate image to yield a first up-sampled image to which is added decompressed error data in a first enhancement layer contained in said embedded bit stream; and

(iii) if said bit stream contains at least three spatial resolutions, a third image whose spatial resolution is higher than said second image by up-sam-pling said first up-sampled image to yield a second intermediate image to which is added decompressed error data in a second enhancement layer contained in said embedded bit stream;

wherein said a viewable image is decodable by said decoder from codewords received in said embedded bit stream that contain less than a number of bits representing a full-length codeword.