Method and apparatus for streaming scalable video

• US 7,095,782 B1
• Filed: 03/01/2000
• Issued: 08/22/2006
• Est. Priority Date: 03/01/2000
• Status: Expired due to Term

• Current Assignee: Funai Electric Co., Ltd.
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Radha, Hayder, Parthasarathy, Kavitha, Cohen, Robert A.
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US09/516,035
• Time in Patent Office: 2,365 Days
• Field of Search: 375/240.01, 375/240.05, 375/240.11, 375/240.12, 375/240.15, 375/240.22, 348/385.1, 348/405.1, 348/390.1, 348/404.1, 348/410.1, 348/412.1, 340/825.5, 370/455, 370/440, 370/468
• US Class Current: 375/240.01
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/238 Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams hybrid fiber coaxial [HFC] networks for downstream channel allocation for video distribution H04L12/2801; flow control in packet networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2404 Monitoring of server processing errors or hardware failure error or fault detection G06F11/07; monitoring in general G06F11/30
• CPC Class Codes: H04N 21/26216 involving the channel capacity, e.g. network bandwidth admission control, resource allocation in open networks H04L12/5692; flow control in packet networks H04L47/10; establishing a schedule or organising the servicing of application requests taking into account QoS H04L67/61
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities

Abstract

A system and method are disclosed for streaming scalable video data over a variable-bandwidth network such as a packet-based one. In other words, the number of bits (for FGS) or sub-layers (for discrete multi-layer scalability) is chosen to be transmitted for each enhancement layer frame. Further, these bits or sub-layers are also partitioned over multiple packets if being sent over a packet-based network. Further, a windowing function is used to smooth the variation in the number of bits or sub-layers transmitted for each frame in situations where the network bandwidth is varying.

Claims

1. A method for streaming scalable video including base layer frames and enhancement layer frames, comprising the steps of:

transmitting at least a portion of at least one of the base layer data frames for a given time interval;

determining if a loss of bandwidth for the given time interval has occurred;

selecting a number of enhancement layer frames to distribute the loss of bandwidth over;

reducing a size of the selected number of enhancement layer frames to accommodate the loss of bandwidth transmit; and

transmitting at least a portion of at least one of the enhancement layer frames for the given time interval, the at least one transmitted enhancement layer frame having been reduced in size.

2. The method according to claim 1, further comprising:

transmitting non-enhancement layer data during the given time interval.

3. The method according to claim 1, wherein the reducing step is performed so that the loss of bandwidth is distributed evenly over the selected number of enhancement layer frames.

4. The method according to claim 1, further comprising the steps of:

Determining if bandwidth remains in the given time interval; and

if bandwidth remains in the given time interval, transmitting at least a portion of one or more enhancement layer frame from a second given time interval in the given time interval.

6. The method according to claim 1, wherein enhancement layer data in the enhancement layer frames has a fine grain scalability structure.

12. The method according to claim 1, wherein the enhancement layer frames over which the loss of bandwidth is distributed comprise frames within the given time interval.

13. The method according to claim 1, wherein the reducing step comprises:

calculating an amount of enhancement layer data accommodating the loss of bandwidth during the given time interval.

14. The method according to claim 1, wherein the step of determining if a loss of bandwidth has occurred for the given time interval comprises:

determining a number of bits during the given time interval consumed by transmission of non-enhancement layer data.

15. The method according to claim 1, wherein the step of determining if a loss of bandwidth has occurred for the given time interval comprises:

determining a number of bits during the given time interval lost due to at least one of packet loss, noise, and bandwidth variation.

16. The method according to claim 1, wherein the reducing step comprises:

calculating a number of lost bandwidth bits to be allocated to each of the selected number of enhancement layer frames.

17. The method according to claim 1, wherein the step of transmitting at least a portion of at least one of the enhancement layer frames comprises:

transmitting a first reduced amount of enhancement layer data in first and last frames of the selected number of enhancement layer frames; and

transmitting a second reduced amount of enhancement layer data different from the first amount in an enhancement layer frame between the first and last enhancement layer frames.

18. The method according to claim 1, wherein the determining step, selecting step, reducing step, and second transmitting step cumulatively result in dynamic adaptation of the scalable video stream to temporary reductions in available bandwidth during transmission of a portion of the scalable video stream.

19. The method according to claim 1, wherein the step of selecting the number of enhancement layer frames to distribute the loss of bandwidth over comprises:

selecting a predetermined number of remaining frames to be transmitted during the given time interval.

20. The method according to claim 1, further comprising:

following transmission of at least one of the enhancement layer frames that have been reduced in size, resuming transmission of enhancement layer frames that have not been reduced in size.

21. The method of claim 1, further comprising:

determining if the selected number of enhancement layer frames has expired;

determining if any left-over enhancement layer data exists;

selecting a second number of enhancement layer frames to distribute the left-over enhancement data over;

reducing a size of the selected second number of enhancement layer frames to accommodate the left-over enhancement data; and

transmitting at least one of the second number of enhancement layer frames in a second given time interval.

22. The method of claim 1, wherein reducing the size of the selected number of enhancement layer frames comprises reducing the size of at least two enhancement layer frames by different amounts.

5. A method for streaming scalable video including base layer data and enhancement layer data, comprising the steps of:

transmitting the base layer data for a given interval within a plurality of time intervals for a single video stream;

determining if a loss of bandwidth has occurred during the given interval;

selecting a predetermined number of frames to distribute the loss of bandwidth over;

calculating a reduced amount of enhancement layer data to transmit in the predetermined number of frames;

transmitting the reduced amount of enhancement layer data during the given interval;

determining if the predetermined number of frames has expired;

determining if any left-over enhancement layer data exists;

selecting a second predetermined number of frames to distribute the left-over enhancement data over;

calculating a second reduced amount of enhancement layer data to transmit in the second predetermined number of frames; and

transmitting the second reduced amount of enhancement layer data in a second given interval.

7. A method for streaming scalable video including base layer frames and enhancement layer frames, comprising the steps of:

transmitting at least a portion of at least one of the base layer frames for a given time interval;

selecting a number of enhancement layer frames if a loss of bandwidth has occurred for the given time interval;

distributing the loss of bandwidth over the selected number of enhancement layer frames by reducing a size of the selected number of enhancement layer frames; and

transmitting at least a portion of at least one of the enhancement layer frames for the given time interval, the at least one transmitted enhancement layer frame having been reduced in size.

8. The method according to claim 7, wherein the distributing step is performed so that the loss of bandwidth is distributed evenly over the selected number of enhancement layer frames.

9. A memory medium including code for streaming scalable video including base layer frames and enhancement layer frames, the code comprising:

first transmitting code for transmitting at least a portion of at least one of the base layer frames for a given time interval;

determining code for determining if a loss of bandwidth has occurred for the given time interval;

selecting code for selecting a number of enhancement layer frames to distribute the loss of bandwidth over;

calculating code for calculating a reduced size of the selected number of enhancement layer frames; and

second transmitting code for transmitting at least a portion of at least one of the enhancement layer frames for the given time interval, the at least one transmitted enhancement layer frame having been reduced in size.

10. An apparatus for streaming scalable video including base layer frames and enhancement layer frames, comprising:

a memory which stores executable code; and

a processor which executes the code stored in the memory so as to;

transmit at least a portion of at least one of the base layer frames for a given time interval,

determine if a loss of bandwidth has occurred for the given time interval,

select a number of enhancement layer frames within the given time interval over which to distribute the loss of bandwidth,

calculate a reduced size of the selected number of enhancement layer frames to accommodate the loss of bandwidth, and

transmit at least a portion of at least one of the enhancement layer frames for the given time interval, the at least one transmitted enhancement layer frame having been reduced in size.

11. An apparatus for streaming scalable video including base layer frames and enhancement layer frames, comprising:

means for transmitting at least a portion of at least one of the base layer data frames for a given time interval;

means for determining if a loss of bandwidth has occurred for the given time interval;

means for selecting a number of enhancement layer frames to distribute the loss of bandwidth over;

means for reducing a size of the selected number of enhancement layer frames to accommodate the loss of bandwidth; and

means for transmitting at least a portion of at least one of the enhancement layer frames for the given time interval, the at least one transmitted enhancement layer frame having been reduced in size.