Techniques for adaptive video streaming

• US 9,571,827 B2
• Filed: 09/28/2012
• Issued: 02/14/2017
• Est. Priority Date: 06/08/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Su, Yeping, Wu, Hsi-Jung, Pan, Hao, Zhang, Ke
• Primary Examiner(s): Anyikire, Chikaodili E
• Assistant Examiner(s): KALAPODAS, DRAMOS
• Application Number:
  US13/631,605
• Publication Number:
  US 20130329781A1
• Time in Patent Office: 1,600 Days
• Field of Search: 725/95, 725/96, 725/100
• US Class Current: 1/1
• CPC Class Codes: H04N 19/10 using adaptive coding
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

A video coding server may code a common video sequence into a plurality of coded data streams, each coded data stream representing the video sequence coded using coding parameters tailored for a respective transmission bit rate. The coding may cause a set of transmission units from among the coded data streams to include coded video data from a common point of the video sequence, and a first coded frame of each transmission unit of the set to be a synchronization frame. A manifest file may be built representing an index of transmission units of the respective coded data streams. The coded data streams and manifest file may be stored by the server for delivery to a client device. During download and decode, the chunks may be decoded efficiently even when switching among streams because the first frame in each chunk is a synchronization frame.

Claims

1. A video coding method, comprising:

coding a common video sequence into a plurality of coded data streams, each coded data stream including a base layer of coded data representing the video sequence coded using coding parameters tailored for a respective transmission bit rate of the respective stream, at least one coded stream including at least one enhancement layer of coded data representing scalable coding of the video sequence with reference to a transmission bit rate associated with the base layer of a next higher stream; and

building a manifest file representing an index of transmission units of the respective coded data streams;

wherein;

sets of transmission units from among the coded base layers and enhancement layers respectively contain multiple frames and are aligned to each other, and

a first coded frame of each transmission unit of the base layer streams is coded as to be a synchronization frame, the synchronization frame defining a path such that all coded frames of the respective transmission unit are decoded according to the path; and

the coded data streams and manifest file are stored for delivery to a client device.

2. The method of claim 1, wherein the synchronization frame of each transmission unit is decodable without reference to any previously-coded frame of the video sequence.

3. The method of claim 1, wherein at least two coded data streams are coded at different frame rates.

4. The method of claim 1, wherein at least two coded data streams are coded at different frame resolutions.

5. The method of claim 1, wherein chunks of a common coded data stream include video data coded at different frame resolutions.

6. The method of claim 1, wherein the synchronization frame is an I-frame.

7. The method of claim 1, wherein the synchronization frame is an Instantaneous Decoder Refresh Frame.

8. A video streaming method, comprising:

transmitting to a client device transmission units retrieved from a first stored sequence of base layer coded video data of a requested video that has been coded using a first selection of coding parameters tailored for a transmission bit rate of the first stored sequence of base layer coded video data,

in response to a request from the client device,

transmitting to the client device transmission units of enhancement layer coded video data representing scalable coding with reference to a transmission bit rate associated with a second stored sequence of base layer coded video data, and

transmitting to the client device transmission units retrieved from the second stored sequence of base layer coded video data of the requested video that has been coded using a second selection of coding parameters tailored for the transmission bit rate associated with the second stored sequence of base layer coded video data,

wherein the stored transmission units of the first and second sequences include multiple frames and are aligned to each other, and include transmission units that represent common portions of the requested video and include a synchronization frame as a first coded frame therein, the common portions being determined based on a cost of coding the requested video.

9. The method of claim 8, wherein the synchronization frame is an I-frame.

10. The method of claim 8, wherein the synchronization frame is an Instantaneous Decoder Refresh Frame.

11. The method of claim 8, further comprising, prior to transmission of any transmission units, transmitting to the client device a manifest file representing an index of transmission units of the first and second sequences.

12. A video coding method, comprising:

coding a common video sequence into two or more base layer coded data streams, each base layer coded data stream representing the video sequence coded using coding parameters tailored for a respective transmission bit rate, one transmission bit rate higher than another transmission bit rate;

coding the video sequence as at least one enhancement layer coded data stream using the lower bit rate base layer coded data stream as a reference, and the high transmission bit rate as a target bit rate, the at least one enhancement layer coded data stream representing scalable coding of the common video sequence with reference to the high transmission bit rate; and

partitioning, based on a cost of coding the video sequence, the coded data streams into a plurality of transmission units for delivery,

wherein the transmission units of each coded data stream has a counterpart in the other coded data stream,

each transmission unit has multiple frames and the transmission units are aligned to each other, and

the coding steps and the partitioning step cooperatively define continuous decode paths for the coded video sequence when the transmission units are decoded in coding order such that all coded frames in the base layer coded data streams are displayed and decoding of transmission units of the enhancement layer coded data stream provides transition between the base layer coded data streams.

13. The method of claim 12, wherein a synchronization frame of each transmission unit is decodable without reference to any previously-coded frame of the video sequence.

14. The method of claim 12, wherein, for at least one coded data stream, the coding comprises coding the video sequence as a base layer of coded data and an enhancement layer of coded data, wherein a first coded frame of each transmission unit of the base layer coded data streams includes a synchronization frame.

15. The method of claim 12, wherein at least two coded data streams are coded at different frame rates.

16. The method of claim 12, wherein at least two coded data streams are coded at different frame resolutions.

17. The method of claim 12, wherein chunks of a common coded data stream include video data coded at different frame resolutions.

18. The method of claim 13, wherein the synchronization frame is an I-frame.

19. The method of claim 13, wherein the synchronization frame is an Instantaneous Decoder Refresh Frame.

20. The method of claim 12, further comprising, for one of the coded data streams, coding the video sequence as an enhancement layer for the one coded data stream.

30. The method of claim 12, further comprising determining a resolution and frame-rate of each transmission unit to maintain a maximum visual quality measured over a decode path.

21. A video decoding method, comprising:

retrieving first transmission units representing base layer coded video data streams of a video sequence coded for a first delivery rate;

decoding coded data present in the transmission units of the first delivery rate;

requesting second transmission units representing base layer coded video data streams of the video sequence coded for a second delivery rate;

in response to the request;

receiving transmission units of enhancement layer coded video data representing scalable coding with reference to the second delivery rate; and

thereafter, receiving and decoding the second transmission units;

wherein a first coded frame of each transmission unit of the first and second transmission units is a synchronization frame, and

wherein the transmission units of the first and second transmission units include multiple frames and are aligned to each other, and include transmission units that represent common portions of the video sequence, the common portions being determined based on a cost of coding the video sequence.

22. The method of claim 21, wherein the synchronization frame of each transmission unit is decodable without reference to any previously-coded frame of the video sequence.

23. The method of claim 21, wherein, for at least one coded data stream, the coding comprise coding the video sequence as a base layer of coded data and an enhancement layer of coded data, wherein the base layer includes the synchronization frame.

24. The method of claim 21, wherein at least two coded data streams are coded at different frame rates.

25. The method of claim 21, wherein at least two coded data streams are coded at different frame resolutions.

26. The method of claim 21, wherein chunks of a common coded data stream include video data coded at different frame resolutions.

27. The method of claim 21, wherein the synchronization frame is an I-frame.

28. The method of claim 21, wherein the synchronization frame is an Instantaneous Decoder Refresh Frame.

29. The method of claim 21, wherein each coded data stream comprises data indicating its visual quality, which is used to decode the stream.