Continuously adaptive digital video compression system and method for a web streamer

• US 6,091,777 A
• Filed: 05/26/1998
• Issued: 07/18/2000
• Est. Priority Date: 09/18/1997
• Status: Expired due to Fees

• Current Assignee: Systems Technology Incorporated
• Original Assignee: Cubic Video Technologies Incorporated
• Inventors: Guetz, William N., Bakhru, Keshoolal, Ritchey, Fredric W.
• Primary Examiner(s): Lee, Richard
• Assistant Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US09/084,820
• Time in Patent Office: 784 Days
• Field of Search: 348/384, 348/385, 348/415, 348/413, 348/402, 348/390, 348/10, 348/14, 348/15, 348/17, 348/12, 348/398, 348/416, 382/232, 382/244, 382/239, 382/246, 382/250, 382/242, 395/200.77, 395/672, 345/336, 345/337, 341/65, 370/354, 370/468, 375/240
• US Class Current: 375/240.11
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/635 characterised by filter definition or implementation details
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 19/93 Run-length coding

Abstract

A cost effective continuously adaptive digital video system and method for compressing color video data for moving images. The method describes capturing an analog video frame and digitizing the image into a preferred source input format for compression using a combination of unique lossy and lossless digital compression techniques including subband coding, wavelet transforms, motion detection, run length coding and variable length coding. The system includes encoder and decoder (CODEC) sections for compression and decompression of visual images to provide high compression with good to excellent video quality. The compressed video data provides a base video layer and additional layers of video data that are multiplexed with compressed digital audio to provide a data stream that can be packetized for distribution over inter or intranets, including wireless networks over local or wide areas. The (CODEC) system continuously adjusts the compression of the digital images frame by frame in response to comparing the available bandwidth on the data channel to the available bandwidth on the channel for the previous frame to provide an output data stream commensurate with the available bandwidth of the network transmission channel and with the receiver resource capabilities of the client users. The compression may be further adjusted by adjustment of the frame rate of the output data stream.

Claims

1. A method for compressing input color video data for moving images for a web stream to be transmitted over a transmission channel having a variable bandwidth available to client users having receivers that receive data input at different acceptable data rates, which method comprises the steps of:

compressing an input color video data stream having a determinable frame rate for moving images on the basis of a target bandwidth on a data transmission channel having a variable bandwidth to provide a compressed output video data stream having a determinable frame rate;

transmitting the compressed output color video data stream to at least one receiver over said transmission channel;

determining the actual available bandwidth of the transmission channel on a frame by frame basis as the compressed output video data stream is transmitted to at least one receiver,

changing the compression of said in put color video data stream in response to comparing the actual available bandwidth on said transmission channel during transmission of the present frame to the available bandwidth on said transmission channel at the time the last frame of said compressed data stream was transmitted thereover whereby the next frame of the subsequent transmission of said output color video data stream is transmitted in packets at a rate commensurate with the actual bandwidth available to a user of said transmission channel at the time of transmission of the previous frame of said data stream,

continually determining the actual available bandwidth of the transmission channel as said compressed output color video data stream is transmitted to at least one receiver, and

continually changing the compression of said output color video data stream in response to a control signal generated from a comparison between the present actual available bandwidth on the data transmission channel and the actual bandwidth on said transmission channel for the immediately previous frame of said data stream whereby the data rate of the packets of said compressed color video data stream is continually adapted to be commensurate with the available variable bandwidth on the transmission channel to dynamically control the packets of said compressed color video data stream, said compression of the input color video data stream includes providing a plurality of levels of subband coding of the input color video data stream, spectrally decomposing selected blocks of pixels of said subband levels to image transform coefficients representative of a level of movement within each pixel block and said control signal is determined by detecting the difference between the present available bandwidth of said output color video data stream and the available bandwidth of the color video data stream transmitted in an immediately previous frame and dynamically changing the rate of compression, as necessary, of said input color video data stream in response to said control signal to provide an output color video data stream in packets having a data rate commensurate with the available bandwidth on the transmission channel to receive such packets.

2. The method of claim 1 which further includes the steps of determining the data rate capability of the receiver of at least one client user to receive said output color video data stream, and

adjusting the data rate of said output color video data stream to be an acceptable data rate for the receiver of the client user.

3. The method of claim 1 wherein the input color video data stream and the output color video data stream is presented in successive frames and the detection of motion is performed by determining whether a threshold of movement within blocks of image transform coefficients of a frame has been exceeded.

4. The method of claim 3 wherein the threshold of movement is provided by two thresholds with a first threshold being determined by the difference in intensity level between corresponding image transfer coefficients in two successive frames and if this difference exceeds said first threshold the image transform coefficient is declared to have moved and with the second threshold being set on the number of image transfer coefficients that have moved within a block of coefficients in a frame, and

the rate of compression of the input color video data stream is controlled by varying the first and second coefficient thresholds in accordance with said control signal to achieve a desired output data rate.

5. The method of claim 3 wherein the data rate of the output color video data stream is further adjusted by adjusting the frame rate of said output data stream on a frame by frame basis whereby said data rate of said output data stream becomes commensurate with the available bandwidth on the transmission channel.

6. The method of claim 1 wherein the data rate of the output color video data stream is determined, in addition to the available bandwidth on the transmission channel, by the receiver capability of a client user in accepting output color video data stream from the transmission channel.

7. The method of claim 6 wherein the data rate of the output color video data stream is determined, in addition to the available bandwidth of the transmission channel, by the least capable receiver resources of a client user of a plurality of users.

8. The method of claim 7 which further includes providing a receiver having a decoder for a client user, which receiver receives the output color video data stream from the transmission channel and decompresses such output color video data stream in real time for suitable display image.

9. The method of claim 8 wherein the compression step provides a base layer output color video data stream that is transmitted over the transmission channel to a plurality of user clients to provide a basic image quality for each user client.

10. The method of claim 9 wherein the compression step further provides a plurality of additional data streams of color video data over said transmission channel to all user clients to permit a receiver of a user client to successively improve the resolution of the received display image and to provide more details for a reconstructed display image.

11. The method of claim 1 which further includes the step of providing a receiver in the form of a suitable digital computer which receives software that is digitally transmitted over the transmission channel to said digital computer for decompressing the received output color video data stream to provide a reconstructed display image.

12. The method of claim 11 which further includes the step of providing a suitable digital computer and operating such computer in accordance with software to provide such data compression step.

13. The method of claim 1 wherein the compression step may be varied to vary the output color video data stream to adjust the resolution, image size, and frame rate for the coded images being transmitted over the transmission channel.

14. The method of claim 1 wherein the data decompressing step may also be varied to vary the received output color video data stream to adjust the resolution, image size and frame rate of the decoded reconstructed visual image to be commensurate with a receiver of a client user receiving such output data stream.

15. The method of claim 1 which further includes the step of multiplexing a compressed digital audio data stream with the output color video data stream.

16. The method of claim 1 wherein the output digital color video data stream is packetized for distribution in an effective network throughput over the transmission channel to a plurality of client users.

17. The method of claim 7 which further provides the step of subjecting portions of the output color video motion data stream to loss-less compression prior to transmission over a transmission channel having a variable bandwidth available to each client user.

18. The method of claim 17 wherein said loss-less compression is provided by run-length encoding and Huffman encoding.

19. A cost effective method for compressing input color video data for moving images for the purpose of streaming a compressed output color video data stream over a packet switched network in the form of a data transmission channel having a variable bandwidth to client users having receivers that provide high quality visual images and which receive data input at different rates, which method comprises the steps of:

compressing an input digital color video data stream having moving images at a predetermined data rate on the basis of a target bandwidth available on a data transmission channel to provide a compressed output color video data stream comprising a plurality of digital video data streams representative of various levels of resolution of a visual image represented in successive video frames and spectrally decomposing the image frames of the input color video signal into a plurality of multilevel frequency subbands and to a plurality of levels and selecting from said frequency subbands a base level digital color video raw data stream that is transmitted in the output video data stream for creation of a base level visual image and at least two additional digital color data streams to provide additional visual details of a visual image as may be desired by a client user;

transmitting the compressed output color video data stream in packets of determinable length to at least one receiver over a data transmission channel;

determining the present bandwidth available on the transmission channel on a frame by frame basis as the compressed output video data streams were transmitted to at least one receiver;

adjusting the rate of compression of said input digital color video data stream in response to such determination on a frame by frame basis whereby the immediately succeeding transmission of a frame of said output digital color video data stream is commensurate with the present bandwidth available on said data transmission channel; and

decompressing the output digital color video data stream at each client user to provide a color visual image having motion.

20. The method of claim 19 wherein the input digital color video data and the output digital color video data stream are presented in successive frames and the adjustment of the rate of compression is made on a frame-by-frame basis in response to a control signal generated by a comparison of the data rate of a color video data stream presently being transmitted for a frame to the immediately previous data rate of the previous frame.

21. The method of claim 19 which further includes providing the compressed output color video data stream to a plurality of client users as a base level visual image is provided to all clients for achieving a basic visual image and providing additional data streams to a client user to successively improve, as may desired by such user, the resolution of the received visual image and to provide more details for the reconstructed visual image.

22. The method of claim 19 which further includes the step of adjusting the resolution, image size and frame rate of the visual image represented by the output color video data stream during the step of compressing the input color video data stream in response to a control signal generated by comparing the actual bandwidth on the transmission channel available to the presently transmitted video to the bandwidth on the transmitted channel available to the last transmitted frame of said video data stream.

23. The method of claim 22 which further includes the step of adjusting the resolution, image size and frame rate of the visual image represented by the output color video data stream during the step of decompressing said output color video data stream in response to a control signal generated by comparing the actual bandwidth on the transmission channel available to the presently transmitted to the bandwidth on the transmission channel available to the last transmitted frame of said video data stream.

24. The method of claim 19 wherein said spectrally decomposing the image frames of the input digital color video signal is performed by wavelet transform filters having outputs that are rounded off to powers of 1/2.

25. The method of claim 19 wherein the compression step includes performing scalar quantization on said additional digital color video data streams to generate quantized additional data streams, performing motion detection on said base level color video raw data stream and said quantized additional digital color video data streams and thereafter performing a loss-less compression coding routine on said quantized additional motion data streams to provide said output color video data stream comprising a motion detected base level data stream and motion detected quantized additional digital color video data streams.

26. The method of claim 25 wherein said loss-less compression coding routine includes run-length coding and Huffman coding.

27. The method of claim 26 wherein the Huffman coding includes generating tables for mapping between pixel values as represented by image transform coefficient values and a code word represented in the output color video data stream.

28. The method of claim 27 wherein said tables generated for the Huffman coding are renewed for each frame to provide optimum distribution of pixel values within the visual image represented by said output color video data stream.

29. The method of claim 19 wherein the rate of compression is also determined by the acceptable data rate of the least capable receiver of a plurality of client users and by a control signal generated by a comparison of the bandwidth of a data stream presently being transmitted for a video frame to the bandwidth on said channel that was available to the immediately preceding video frame of such data stream.

30. The method of claim 19 wherein the data compressing and decompressing steps are provided by data processing means that are operated by software.

31. The method of claim 19 wherein the decompressing step is performed in real time by a receiver of a client user without storage of such received color video data stream.

32. The method of claim 19 which further includes the step of multiplexing a compressed digital audio data stream with the output color video data stream.

33. The method of claim 32 wherein said multiplexed output digital video data stream is packetized as effective network throughput for distribution over a data transmission channel providing a packet switched network to a plurality of client users.

34. Cost effective method for compressing input color video data for moving images for the purpose of streaming a compressed output color video data stream over a packet switched network in the form of a data transmission channel having a variable bandwidth to client users having receivers that provide high quality visual images and which may receive data input at different acceptable rates, which method comprises the steps of:

compressing an input digital color video data stream having moving images at a predetermined data rate on the basis of a target bandwidth available on a data transmission channel to provide a compressed output color video data stream comprising a plurality of digital video data streams representative of various levels of resolution of a visual image represented in successive video frames and performing scalar quantization on said additional digital color video data streams to generate quantized additional data streams, performing motion detection on said base level color video raw data stream and said quantized additional digital color video data streams and thereafter performing a loss-less compression coding routine on said quantized additional motion data streams to provide said output color video data stream comprising a motion detected base level data stream and motion detected quantized additional digital color video data streams;

transmitting the compressed output color video data stream in packets of determinable length to at least one receiver over a data transmission channel;

determining the present bandwidth available on the transmission channel on a frame by frame basis as the compressed output video data streams were transmitted to at least one receiver;

adjusting the rate of compression of said input digital color video data stream in response to such determination on a frame by frame basis whereby the immediately succeeding transmission of a frame of said output digital color video data stream is commensurate with the present bandwidth available on said data transmission channel; and

decompressing the output digital color video data stream at each client user to provide a color visual image having motion.