Software-based encoder for a software-implemented end-to-end scalable video delivery system

• US 5,768,535 A
• Filed: 04/08/1997
• Issued: 06/16/1998
• Est. Priority Date: 04/18/1995
• Status: Expired due to Term

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Northcutt, J. Duane, Hanko, James G., Chaddha, Navin, Wall, Gerard A.
• Primary Examiner(s): Voeltz, Emanuel Todd
• Assistant Examiner(s): Peeso, Thomas
• Application Number:
  US08/835,855
• Time in Patent Office: 434 Days
• Field of Search: 395/200.77, 395/200.32, 348/403, 348/420, 370/394, 370/473, 370/352, 370/355, 382/232, 382/233, 382/235
• US Class Current: 709/247
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/39 involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/94 Vector quantisation
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/440227 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/631 Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
• CPC Class Codes: H04N 21/6405 Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16
• CPC Class Codes: H04N 7/17336 Handling of requests in head-ends
• CPC Class Codes: H04N 7/24 Systems for the transmission of television signals using pulse code modulation H04N21/00 takes precedence

Abstract

A software-based encoder is provided for an end-to-end scalable video delivery system that operates over heterogeneous networks. The encoder utilizes a scalable video compression algorithm based on a Laplacian pyramid decomposition to generate an embedded information stream. The decoder decimates a highest resolution original image, e.g., 640×480 pixels, to produce an intermediate 320×240 pixel image that is decimated to produce an intermediate 160×120 pixel image that is compressed to form an encodable base layer 160×120 pixel image. This base layer image is decompressed to form an image that is up-sampled by interpolation to produce an up-sampled 320×240 pixel image. This up-sampled image is subtracted from the intermediate 320×240 pixel image to form an error image that is compressed and encoded as a first enhancement 640×480 pixel layer. The decompressed base layer image is also up-sampled at step to produce an up-sampled 640×480 pixel image that is subtracted from the original 640×480 pixel image 200 to yield an error image that is compressed to yield a second enhancement 320×240 pixel layer. Collectively, the base and enhancement layers comprise the transmitted embedded bit stream. At the receiving end, the decoder extracts from the embedded stream different streams at different spatial and temporal resolutions. Because decoding requires only additions and look-ups from a small stored table, decoding occurs in real-time.

Claims

1. A scalable video encoder for use in a video delivery system server having a video source whose video images have a first spatial resolution, the scalable video encoder providing a decodable embedded bit stream that is network transmittable and includes image data at at least two spatial resolutions, the encoder including:

a computer memory storing an executable software routine programmed to receive and process a first image from said video source to form at least a base image having resolution less than said first image, and to form a first enhancement layer image having resolution less than said first image but greater than said base layer image;

said decodable embedded bit stream comprising data packets of fixed length code words and containing at least said base layer image and said first enhancement layer image.

2. The encoder of claim 1, wherein before being encoded, said data packets are converted into a frequency domain.

3. The encoder of claim 1, wherein said data packets are encoded using quantized indexable representations of discrete cosine transformed video image data.

4. The encoder of claim 3, wherein encoding uses discrete cosine transform coefficients that include input-weighted squared error defined as follows:

##EQU2## where y_j and y_j are components of a transformed vector y and of a corresponding reproduction vector y, and where w_j is a component of a weight vector generally dependent only upon y.

5. The encoder of claim 1, wherein said embedded bit stream includes information for at least two spatial resolutions and for at least one of said two spatial resolutions frequency domain information is vector quantized.

6. The encoder of claim 5, wherein said vector quantization is tree-structured such that vector quantization has a tree depth R and a vector dimension k;

and

wherein bitstream bit rates O/k . . . , R/k are provided for said embedded bit stream.

7. The encoder of claim 5, wherein said vector quantization is tree-structured and includes a perception model.

8. The encoder of claim 1, wherein said encoder encodes spatial resolution data using a discrete cosine transformation followed by a tree-structured vector quantization upon results of said transformation.

9. The encoder of claim 1, wherein said encoder forms said base layer image by:

decimating said first image to form a first intermediate image having half said highest resolution;

decimating said first intermediate image to form a second intermediate image that is compressed to form said base layer image.

10. The encoder of claim 9, wherein said encoder forms said first enhancement layer image by:

decompressing said base layer image to form a third intermediate image that is interpolated to form a fourth intermediate image that is subtracted from said first intermediate image to form a fifth intermediate image that is compressed to form said first enhancement layer image.

11. The encoder of claim 1, wherein said decodable embedded bit stream includes image data for an additional image having a third spatial resolution that is equal to resolution of said first image.

12. For use in a video delivery system server having a video source whose video images have a first spatial resolution, a method of scalably encoding video to provide a decodable embedded bit stream that is network transmittable and includes image data at at least two spatial resolutions, the method including the following steps:

(a) providing a central processor unit coupled to a computer memory; and

(b) storing an encoding software routine on said computer memory, said routine executable by said central processor unit so as to receive and process a first image from said video source to form at least a base image having resolution less than said first image, and to form a first enhancement layer image having resolution less than said first image but greater than said base layer image;

wherein said decodable embedded bit stream comprises data packets of fixed length code words and contains at least said base layer image and said first enhancement layer image.

13. The method of claim 12, wherein before being encoded, step (b) includes converting said data packets into a frequency domain.

14. The method of claim 12, wherein step (b) includes encoding said data packets using quantized indexable representations of discrete cosine transformed video image data.

15. The method of claim 14, wherein step (b) is carried out using discrete cosine transform coefficients that include input-weighted squared error defined as follows:

##EQU3## where y_j and y_j are components of a transformed vector y and of a corresponding reproduction vector y, and where w_j is a component of a weight vector generally dependent only upon y.

16. The method of claim 14, wherein step (b) includes encoding spatial resolution data using a discrete cosine transformation followed by a tree-structured vector quantization upon results of said transformation.

17. The method of claim 14, wherein at step (b), said base layer image is formed by:

(b-1) decimating said first image to form a first intermediate image having half said highest resolution; and

(b-2) decimating said first intermediate image to form a second intermediate image that is compressed to form said base layer image.

18. The method of claim 17, wherein at step (b), said first enhancement layer image is formed by:

decompressing said base layer image to form a third intermediate image that is interpolated to form a fourth intermediate image that is subtracted from said first intermediate image to form a fifth intermediate image that is compressed to form said first enhancement layer image.

19. The method of claim 14, wherein said decodable embedded bit stream includes image data for an additional image having a third spatial resolution that is equal to resolution of said first image.

20. The method of claim 12, wherein at step (b) said embedded bit stream includes information for at least two spatial resolutions and for at least one of said two spatial resolutions frequency domain information is vector quantized.

21. The method of claim 20, wherein at step (b), said vector quantization is tree-structured such that vector quantization has a tree depth R and a vector dimension k;

and

wherein bitstream bit rates O/k, . . . , R/k are provided for said embedded bit stream.

22. The method of claim 20, wherein at step (b), said vector quantization is tree-structured and includes a perception model.