Bitstream Switching in Multiple Bit-Rate Video Streaming Environments

• US 20080046939A1
• Filed: 07/26/2006
• Published: 02/21/2008
• Est. Priority Date: 07/26/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Shipeng, Lu, Yan, Wu, Feng, Sun, Wen
• Granted Patent: US 8,311,102 B2
• Time in Patent Office: Days
• US Class Current: 725/90
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip

Abstract

Bitstream switching for multiple bit rate video streaming uses a first algorithm to generate multiple encoded bitstreams of a piece of video at different bit rates. A different algorithm is used to generate multiple side bitstreams of the video. Each side stream is generated at respective ones of the different bit rates. The side bitstreams comprise switching frames to switch from a first bitstream encoded with the first algorithm at a first bit rate to a second bitstream encoded with the first algorithm at a different bit rate. At non-bitstream switching points, frames from respective ones of bitstreams generated with the first algorithm are streamed for presentation. At a bitstream switching point, a switching frame from a respective side bitstream is transmitted to a decoder to transition streaming from a first of the respective ones of the bitstreams to a second of the respective ones of the bitstreams.

Claims

1. A method at least partially implemented by a computer, the method comprising:

generating, using a first encoding algorithm, multiple encoded bitstreams of a same piece of video content, each encoded bitstream of the encoded bitstreams being encoded at a different bit rate;

creating, using a second encoding algorithm that is different than the first encoding algorithm, multiple encoded side bitstreams of the same piece of video content, each encoded side bitstream being encoded at a different bit rate and comprising switching frames generated from frames of the same piece of video content and corresponding to frames of respective ones of the encoded bitstreams;

streaming, at non-bitstream switching points, frames from an encoded bitstream generated with the first encoding algorithm to a client;

switching, at a bitstream switching point, from the encoded bitstream to a different bitstream of the encoded bitstreams by;

streaming a switching frame from a side bitstream encoded with the second algorithm to the client, the switching frame corresponding to a frame in the second bitstream; and

streaming at least a portion of a frame from the bitstream switching point from the different bitstream and subsequent frames from the different bitstream to the client; and

wherein receipt by the client of frames from the encoded bitstream, the switching frame, and frames from the second bitstream causes the client to reconstruct respective frames of the same piece of video content for presentation to a user.

2. The method of claim 1, wherein the first encoding algorithm is an H.264 encoding algorithm.

3. The method of claim 1, wherein the second encoding algorithm is a Wyner-Ziv encoding algorithm.

4. The method of claim 1, wherein receipt of at least the switching frame provides the client with information to construct a bitstream transition frame, the bitstream transition frame representing a first reconstructed frame from the different bitstream to be presented to the user.

5. The method of claim 1, wherein the switching represents an operation to switch from a bitstream encoded at a first bit rate to a second bitstream encoded at a second bit rate that is greater than the first bit rate.

6. The method of claim 1, wherein the switching represents an operation to switch from a bitstream encoded at a first bit rate to a second bitstream encoded at a second bit rate that is less than the first bit rate.

7. The method of claim 1, wherein the frames from the encoded bitstream are for decoding by a first decoding algorithm, the switching frame is for decoding by a second decoding algorithm that is different from the first decoding algorithm, and frames from the different bitstream are for decoding by the first decoding algorithm.

8. The method of claim 1:

wherein the switching frame represents parity bits from a frame at time n−

1 in a timeline of the same piece of video content, the switching frame being encoded at a same bit rate used to generate the different bitstream, n being the bitstream switching point; and

wherein a bitstream transition frame is generated by the client from information provided by a reconstructed reference frame corresponding to time n−

1 of the first bitstream and the switching frame

9. The method of claim 1:

wherein the switching frame represents parity bits from a frame at time n in a timeline of the same piece of video content, the switching frame being encoded at a same bit rate used to generate the different bitstream, n being the bitstream switching point;

wherein the at least a portion is motion compensation information of the frame from time n from the different bitstream; and

wherein the respective frames comprise a transition frame to transition presentation of frames decoded from the encoded bitstream to frames decoded from the different bitstream, the transition frame being generated based on the motion vector information, a reconstructed previous frame associated with time n−

1 in the first bitstream, and decoded intra blocks of a frame associated with time n in the second bitstream, n being the bitstream switching point.

10. The method of claim 1, wherein switching from the encoded bitstream to a different bitstream is performed responsive to detecting network data throughput bandwidth conditions incompatible with a bit rate used to encode the encoded bitstream and compatible with a bit rate used to encode the different bitstream.

11. A computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions implementing, when executed, operations for:

receiving a set of frames associated with a current bitstream;

decoding the frames with a first decoding algorithm for presentation to a user;

receiving, at a switching point n, a switching frame generated from a piece of video content encoded at a different bit rate than the current bitstream;

responsive to receiving the switching frame;

decoding the switching frame to generate a transition frame, the decoding being performed with a second decoding algorithm using at least a reconstructed frame as side information, the reconstructed frame corresponding to a frame from the current bitstream, the second decoding algorithm being different than the first decoding algorithm, the second decoding algorithm being different than the first decoding algorithm; and

presenting the transition frame to the user; and

receiving a set of frames from the different bitstream for decoding using the first decoding algorithm and subsequent presentation to the user.

12. The computer-readable medium of claim 11, wherein the first decoding algorithm is an H.264 encoding algorithm.

13. The computer-readable medium of claim 11, wherein the second decoding algorithm is a Wyner-Ziv encoding algorithm.

14. The computer-readable medium of claim 11, wherein receipt of the switching frame provides information to construct a bitstream transition frame, the bitstream transition frame representing a first reconstructed frame from the different bitstream to be presented to the user.

15. The computer-readable medium of claim 11, further comprising:

receiving only motion compensation information for a frame of the different bitstream, the frame corresponding to frame of the piece of video content at time n; and

wherein the motion compensation information and the switching frame in view of the reconstructed frame provides information to construct a bitstream transition frame for presentation to the user when transitioning between the current bitstream and the different bitstream.

16. The computer-readable medium of claim 11, wherein the switching frame represents parity bits from a frame at time n−

1 in a timeline of the piece of video content, the switching frame being encoded at a same bit rate used to generate the different bitstream.

17. The computer-readable medium of claim 11, wherein the switching frame represents parity bits from a frame at time n in a timeline of the same piece of video content, the switching frame being encoded at a same bit rate used to generate the different bitstream.

18. The computer-readable medium of claim 11, wherein switching from the current bitstream to a different bitstream is performed responsive to detecting network data throughput bandwidth conditions incompatible with a bit rate used to encode the current bitstream and compatible with a bit rate used to encode the different bitstream.

19. A computing device comprising:

a processor; and

a memory coupled to the processor, the memory comprising computer-program instructions executable by the processor for;

encoding, using Wyner-Ziv encoding, video content to generate a bitstream of switching frames at a particular bit rate;

encoding, using an encoding algorithm different from Wyner-Ziv, frames of the video content at the particular bit rate and at a different bit rate;

streaming frames encoded at the different bit rate for decoding and presentation to a user;

determining that network conditions are incompatible with streaming the frames but compatible with frames encoded at the particular bit rate;

responsive to the determining;

transmitting, at a switching point, a switching frame from the bitstream of switching frames for reconstructing a first frame from frames encoded at the particular bit rate with a non-Wyner-Ziv encoding, the first frame for presentation to the user; and

streaming respective ones of the frames encoded at the particular bit rate with a non-Wyner-Ziv encoding for presentation to the user, the frames being at least subsequent in a video presentation timeline to the switching point.

20. The computing device of claim 19, wherein the encoding algorithm different from Wyner-Ziv is an H.264 encoding algorithm.