Method for transmitting video information over a communication channel

• US 6,519,004 B1
• Filed: 10/09/1998
• Issued: 02/11/2003
• Est. Priority Date: 10/09/1998
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bahl, Paramvir
• Primary Examiner(s): Philippe, Gims S.
• Application Number:
  US09/169,120
• Time in Patent Office: 1,586 Days
• Field of Search: H04/N.712, 348/384.1, 348/422.1, 348/414.1, 348/385.1, 348/387.1, 348/388.1, 348/390.1, 348/389.1, 348/404.1, 348/408.1, 348/423.1, 348/405, 348/498, 348/403, 382/239, 375/240.05, 375/240.11, 375/240.01, 370/468, 370/458, 710/107
• US Class Current: 348/385.1
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/29 involving scalability at the object level, e.g. video object layer [VOL]
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/2362 Generation or processing of Service Information [SI]
• CPC Class Codes: H04N 21/23655 Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/440227 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/6131 involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/631 Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
• CPC Class Codes: H04N 21/6375 : for requesting retransmission , e.g. of data packets lost or corrupted during transmission from serverARQ protocols H04L1/18; implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] H04L69/16

Abstract

An adaptive region-based, multi-scale, motion compensated video compression algorithm design for transmission over hostile communication channels. The algorithm is embodied in a video encoder that extracts spatial information from video frames to create video regions that are then decomposed into sub-bands of different perceptual importance before being compressed and transmitted independently. The system further uses unequal error protection, prioritized transmission and reconstruction to guarantee a minimum spatial and temporal resolution at the receiver. In particular, the region segmented frames bound both spatial and temporal error propagation within frames. A connection-level inter-region statistical multiplexing scheme is also employed to ensure optimal utilization of reserved transmission bandwidth.

Claims

1. A method of requesting bandwidth in a communications channel for use in transmitting a plurality of video frames in a video segment to a receiver, comprising the steps of:

spatially segmenting each video frame in the video segment;

determining the number of bits required to transmit a predefined subset of each spatially segmented video frame;

with respect to all of the plurality of video frames, determining the maximum number of bits required to transmit the predefined subset of any video frame of the plurality of video frames; and

statically requesting an amount of bandwidth from the communications channel for the transmission of all of the plurality of frames as a function of the maximum number of bits required to transmit any frame of the plurality of frames whereby a minimum spatial and temporal resolution is guaranteed at the receiver.

2. The method as recited in claim 1 wherein the step of statically requesting an amount of bandwidth comprises the further steps of multiplying the maximum number of bits required to transmit the predefined subset of any video frame of the plurality of video frames by a desired target video frame display rate and using the result to statically request the amount of bandwidth at a time of connection.

3. The method as recited in claim 2 further comprising the step of compressing each segment generated by the step of spatially segmenting before the step of determining the number of bits required to transmit a predefined subset of each spatially segmented video frame.

4. A computer-readable medium for requesting bandwidth in a communications channel for use in transmitting a plurality of video frames in a video segment to a receiver, the computer-readable medium having instructions for performing the steps of:

spatially segmenting each video frame in the video segment;

determining the number of bits required to transmit a predefined subset of each spatially segmented video frame;

with respect to all of the plurality of video frames, determining the maximum number of bits required to transmit the predefined subset of any video frame of the plurality of video frames; and

statically requesting an amount of bandwidth in the communications channel for the transmission of all of the plurality of frames as a function of the maximum number of bits required to transmit any frame of the plurality of frames whereby a minimum spatial and temporal resolution is guaranteed at the receiver.

5. The computer-readable medium as recited in claim 4 wherein the instructions for performing the step of statically requesting an amount of bandwidth perform the further steps of multiplying the maximum number of bits required to transmit the predefined subset of any video frame of the plurality of video frames by a desired target video frame display rate and using the result to statically request the amount of bandwidth at a time of connection.

6. The computer-readable medium as recited in claim 5 further comprising instructions for performing the step of compressing each segment generated by the step of spatially segmenting before the step of determining the number of bits required to transmit a predefined subset of each spatially segmented video frame.

7. A method for improving the efficiency of a video transmission comprised of a plurality of video frames over a channel in a communications network comprised of a plurality of channel frames, the method comprising the steps of:

segmenting each video frame of the plurality of video frames into a plurality of discrete digitally represented components;

determining the number of bits required to transmit a predefined subset of the discrete digitally represented components of each video frame of the plurality of video frames;

with respect to all of the plurality of video frames, determining the greatest number of bits required to transmit the predefined subset of the discrete digitally represented components of any video frame of the plurality of video frames;

requesting from the channel an amount of bandwidth as a function of the greatest number of bits required to transmit the predefined subset of the discrete digitally represented components of any video frame of the plurality of video frames;

transmitting within the requested bandwidth in each channel frame of the plurality of channel frames the predefined subset of the discrete digitally represented components of sequential ones of the plurality of video frames; and

using any left over bandwidth in each channel frame of the plurality of channel frames to transmit any remaining discrete digitally represented components of the corresponding one of the plurality of video frames.

8. A computer-readable medium for use in improving the efficiency of a video transmission comprised of a plurality of video frames over a channel in a communications network comprised of a plurality of channel frames, the computer-readable medium having instructions for performing the steps of:

segmenting each video frame of the plurality of video frames into a plurality of discrete digitally represented components;

determining the number of bits required to transmit a predefined subset of the discrete digitally represented components of each video frame of the plurality of video frames;

with respect to all of the plurality of video frames, determining the greatest number of bits required to transmit the predefined subset of the discrete digitally represented components of any video frame of the plurality of video frames;

requesting from the channel an amount of bandwidth as a function of the greatest number of bits required to transmit the predefined subset of the discrete digitally represented components of any video frame of the plurality of video frames;

transmitting within the requested bandwidth in each channel frame of the plurality of channel frames the predefined subset of the discrete digitally represented components of sequential ones of the plurality of video frames; and

using any left over bandwidth in each channel frame of the plurality of channel frames to transmit any remaining discrete digitally represented components of the corresponding one of the plurality of video frames.

9. A method of requesting bandwidth in a communications channel for use in transmitting a plurality of video frames in a video segment to a receiver, comprising:

spatially segmenting each video frame in the video segment;

determining the number of bits required to transmit a predefined subset of each spatially segmented video frame;

with respect to all of the plurality of video frames, determining the maximum number of bits required to transmit the predefined subset of any video frame of the plurality of video frames; and

statically requesting an amount of bandwidth from the communications channel for the transmission of all of the plurality of frames as a function of the maximum number of bits required to transmit any frame of the plurality of frames whereby a minimum spatial and temporal resolution is guaranteed at the receiver.

10. A method for improving the efficiency of a video transmission comprised of a plurality of video frames over a channel in a communications network comprised of a plurality of channel frames, the method comprising:

segmenting each video frame of the plurality of video frames into a plurality of discrete digitally represented components;

determining the number of bits required to transmit a predefined subset of the discrete digitally represented components of each video frame of the plurality of video frames;

with respect to all of the plurality of video frames, determining the greatest number of bits required to transmit the predefined subset of the discrete digitally represented components of any video frame of the plurality of video frames;

requesting from the channel an amount of bandwidth as a function of the greatest number of bits required to transmit the predefined subset of the discrete digitally represented components of any video frame of the plurality of video frames;

transmitting within the requested bandwidth in each channel frame of the plurality of channel frames the predefined subset of the discrete digitally represented components of sequential ones of the plurality of video frames; and

using any left over bandwidth in each channel frame of the plurality of channel frames to transmit any remaining discrete digitally represented components of the corresponding one of the plurality of video frames.

11. A method of obtaining the use of communication capacity in a communications channel for transmitting a video sequence comprising a plurality of visual images to a receiver, comprising:

decomposing each visual image in the video sequence into a plurality of component parts;

associating each visual image in the sequence with an amount of data required to transmit a predetermined portion of the component parts of the visual image;

identifying a maximum amount of data required to transmit the predetermined portion of component parts of any image; and

obtaining the use of communication capacity in the communications channel for the transmission of the video sequence based on the identified maximum amount of data required to transmit the predetermined portion of component parts of any image.

12. The method according to claim 11 wherein obtaining the use of communication capacity in the communications channel further comprises multiplying the identified maximum amount of data required to transmit the predetermined portion of component parts of any image by a target frame rate.

13. The method according to claim 12 further comprising coding each component part generated by the step of decomposing each visual image prior to identifying a maximum amount of data required to transmit the predetermined portion of component parts of any image.

14. The method according to claim 11, wherein decomposing each visual image in the video sequence into a plurality of component parts comprises decomposing each visual image into a plurality of different frequency components.

15. The method according to claim 14, wherein the plurality of different frequency components comprise a DC frequency component and a plurality of AC frequency components.

16. The method according to claim 11, wherein decomposing each visual image in the video sequence into a plurality of component parts comprises decomposing each visual image into a plurality of spatial components.

17. A method for improving the efficiency of use of a communication transmission path during transmission of a video series comprising a plurality of images, the method comprising:

segmenting each image of the plurality of images into a plurality of discrete digitally represented components;

determining a number of bits required to transmit a predefined subset of the discrete digitally represented components of each image;

identifying a maximum bit requirement corresponding to the number of bits required to transmit the predefined subset of the discrete digitally represented components of a particular image of the plurality of images requiring a greater number of bits for transmission of its predefined subset of components than any other of the plurality of images;

obtaining authorization to utilize an authorized amount of bandwidth in the communication transmission path, wherein the authorized amount of bandwidth is selected as a fiction of the maximum bit requirement;

transmitting within the authorized amount of bandwidth the predefined subset of the discrete digitally represented components of each image in the video series; and

if a utilized amount of bandwidth consumed to transmit the predefined subset of the discrete digitally represented components of each image in the video series is less than the authorized amount of bandwidth, using the difference in bandwidth between the authorized amount of bandwidth and the utilized amount of bandwidth to transmit at least one remaining discrete digitally represented component of at least one of the plurality of images.

18. A method of securing the use of bandwidth for transmitting at least a portion of each of a plurality of pictures in a video segment to a receiver, comprising:

segmenting each picture in the video segment into a plurality of separate frequency elements;

determining a picture requirement number of bits associated with each picture, wherein each picture requirement number of bits corresponds to a number of bits required to transmit a predefined portion of its associated segmented picture;

identifying the highest picture requirement from among the picture requirements for all pictures in the video segment; and

securing a guaranteed amount of bandwidth from a communications channel for the sequential transmission of at least the predefined portion of each segmented picture of the plurality of pictures over the communications channel, wherein the guaranteed amount of bandwidth is chosen as a function of the identified highest picture requirement.

19. The method according to claim 18, wherein securing a guaranteed amount of bandwidth further comprises multiplying the identified highest picture requirement by a target video rate.

20. The method according to claim 18, further comprising encoding at least one frequency element associated with each segmented picture before identifying the highest picture requirement from among the picture requirements for all pictures in the video segment.