Streaming methods and systems
First Claim
1. A method comprising:
- providing a streaming server configured to stream data; and
at the streaming server, dropping at least portions of an enhancement layer sufficient to forward-shift following portions of the enhancement layer by an amount.
2 Assignments
0 Petitions
Accused Products
Abstract
Various embodiments provide methods and systems for streaming data that can facilitate streaming during bandwidth fluctuations in a manner that can enhance the user experience. In one aspect, a forward-shifting technique is utilized to buffer data that is to be streamed, e.g. an enhancement layer in a FGS stream. Various techniques can drop layers actively when bandwidth is constant. The saved bandwidth can then be used to pre-stream enhancement layer portions. In another aspect, a content-aware decision can be made as to how to drop enhancement layers when bandwidth decreases. During periods of decreasing bandwidth, if a video segment does not contain important content, the enhancement layers will be dropped to keep the forward-shifting of the enhancement layer unchanged. If the enhancement layer does contain important content, it will be transmitted later when bandwidth increases.
-
Citations
92 Claims
-
1. A method comprising:
-
providing a streaming server configured to stream data; and
at the streaming server, dropping at least portions of an enhancement layer sufficient to forward-shift following portions of the enhancement layer by an amount. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method comprising:
-
determining an importance of content that is to be streamed to a client by a steaming server; and
making a decision to forward-shift following portions of the content to the client based on the determined importance of the content. - View Dependent Claims (14, 15, 16)
-
-
17. A method comprising:
-
first dropping at least portions of an enhancement layer sufficient to forward-shift following portions of the enhancement layer by an amount;
determining whether a decrease in available bandwidth has occurred;
responsive to said determining, determining whether the enhancement layer contains important content, and;
if the enhancement layer does not contain important content, second dropping additional portions of the enhancement layer in an attempt to preserve a forward-shifted enhancement layer, otherwise, if the enhancement layer does contain important content, delaying transmission of portions of the enhancement layer so that the delayed portions can be transmitted when the available bandwidth increases. - View Dependent Claims (18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. A method comprising:
-
actively dropping portions of an enhancement layer when available bandwidth is constant to effectively forward shift other portions of the enhancement layer;
analyzing content of the enhancement layer to determine its relative importance; and
if content is determined to not be important, dropping associated portions of the enhancement layer, otherwise, delaying transmission of associated portions of the enhancement layer. - View Dependent Claims (29, 30, 31, 32, 33, 34)
-
-
35. A streaming server comprising:
-
one or more processors;
memory; and
software code embodiment in the memory which, when executed by the one or more processors, cause the one or more processors to drop at least portions of an enhancement layer sufficient to forward-shift following portions of the enhancement layer by an amount. - View Dependent Claims (36, 37, 38)
-
-
39. A streaming server comprising:
-
one or more processors;
memory;
software code embodiment in the memory which, when executed by the one or more processors, cause the one or more processors to;
first drop at least portions of an enhancement layer sufficient to forward-shift following portions of the enhancement layer by an amount;
determine whether a decrease in available bandwidth has occurred;
responsive to determining that a decrease in available bandwidth has occurred, determine whether the enhancement layer contains important content, and;
if the enhancement layer does not contain important content, second drop additional portions of the enhancement layer in an attempt to preserve a forward-shifted enhancement layer, otherwise, if the enhancement layer does contain important content, delay transmission of portions of the enhancement layer so that the delayed portions can be transmitted when the available bandwidth increases. - View Dependent Claims (40, 41, 42, 43, 44, 45)
-
- 46. One or more computer-readable media having computer-readable instructions thereon which, when executed by one or more processors, cause the processors to operate a streaming server by dropping at least portions of an enhancement layer comprising part of an FGS stream sufficient to forward-shift other portions of the enhancement layer by an amount.
-
52. One or more computer-readable media having computer-readable instructions thereon which, when executed by one or more processors, cause the processors to:
-
first drop at least portions of an enhancement layer comprising part of an FGS stream sufficient to forward-shift other portions of the enhancement layer by an amount;
determine whether a decrease in available bandwidth has occurred;
responsive to determining that a decrease in available bandwidth has occurred, determine whether the enhancement layer contains important content, and;
if the enhancement layer does not contain important content, second drop additional portions of the enhancement layer in an attempt to preserve a forward-shifted enhancement layer, otherwise, if the enhancement layer does contain important content, delay transmission of portions of the enhancement layer so that the delayed portions can be transmitted when the available bandwidth increases;
determine that the available bandwidth has increased responsive to determining that the available bandwidth has increased, transmit the delayed portions of the enhancement layer; and
after transmitting the delayed portions, drop additional portions of the enhancement layer sufficient to forward shift other portions of the enhancement layer by an amount. - View Dependent Claims (53, 54)
-
-
55. One or more computer-readable media having computer-readable instructions thereon which, when executed by one or more processors, cause the processors to:
-
actively drop portions of an enhancement layer comprising part of an FGS stream when available bandwidth is constant to effectively forward shift following portions of the enhancement layer;
analyze content of the enhancement layer to determine its relative importance; and
if content is determined to not be important, drop associated portions of the enhancement layer, otherwise, delay transmission of associated portions of the enhancement layer. - View Dependent Claims (56)
-
-
57. A method comprising:
-
providing a streaming server configured to stream data to at least one client, said data comprising one or more enhancement layers;
determining whether portions of the one or more enhancement layers are important by taking into account motion that is embodied in said portions;
making a decision to drop enhancement layer portions based, at least in part, on whether the portions are important. - View Dependent Claims (58, 59, 60, 61, 62, 63)
-
-
64. A method comprising:
-
computing perceived motion energy (PME) values for multiple frames of a video stream;
representing the video stream as a PME value sequence;
segmenting the PME value sequence into individual segments, each segment corresponding to multiple frames;
within each segment, assigning one or more priority levels; and
dropping frames having lower priorities when available bandwidth decreases. - View Dependent Claims (65, 66, 67, 68, 69, 70, 71, 72, 73, 74)
-
-
75. A method comprising:
-
providing one or more models that associate perceived motion energy (PME) values for video segments with individual classes, each class being associated with a score that pertains to human-perceived impairment caused by frame rate decreases;
assigning individual video segments a class in accordance with a segment'"'"'s peak PME value;
mapping the assigned class to an individual priority level; and
using the priority levels to ascertain one or more frames within a video segment to drop. - View Dependent Claims (76, 77, 78, 79)
-
-
80. A system comprising:
a rate controller comprising;
a state machine for implementing perceptual rate adaptation through mode and state transitions;
a virtual buffer model communicatively coupled with and providing feedback to the state machine for describing dynamical buffer filling and draining processes; and
a bandwidth allocation module communicatively coupled with the state machine for allocating bandwidth to frames given the state and mode of the state machine. - View Dependent Claims (81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91)
-
92. A method comprising:
-
providing a state machine for implementing perceptual rate adaptation through mode and state transitions, wherein state machine modes define protected priority levels associated with frames that are to be protected, the state machine being configured to transit modes if bandwidth or stream bit rate changes in the sense of long-term, the state machine comprising a first state associated with a state in which available bandwidth is not fluctuating and in which an attempt is made to maintain a window size associated with a sending frame rate, a second state in which the available bandwidth is insufficient and in which the window size is decreased and a client buffer is used for additional bandwidth, and a third state in bandwidth recovers from the second state and in which the window size is increased by actively dropping unprotected frames;
providing a virtual buffer model communicatively coupled with and providing feedback to the state machine for describing dynamical buffer filling and draining processes, the virtual buffer model comprising an input for the sending frame rate associated with the window size, an output associated with a constant frame consumption rate, and a feedback output provided to the state machine and associated with a number of buffered frames after a given time slot; and
providing a bandwidth allocation module communicatively coupled with the state machine for allocating bandwidth to frames given the state and mode of the state machine.
-
Specification