Adaptive streaming techniques

• US 9,992,499 B2
• Filed: 02/26/2014
• Issued: 06/05/2018
• Est. Priority Date: 02/27/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Su, Yeping, Pan, Hao, Normile, James O., Wu, Hsi-Jung
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Uhl, Lindsay
• Application Number:
  US14/190,880
• Publication Number:
  US 20140241415A1
• Time in Patent Office: 1,560 Days
• Field of Search: None
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

Systems and methods are presented for minimizing the suddenness and immediacy of changes to the video quality perceived by users due to bandwidth fluctuations and transitions between different bitrate streams. A method may include identifying an upcoming bitrate change in a bitstream and a nearest scene cut boundary from sync frame scene cut tags included in the bitstream. The method may include calculating whether waiting until the identified nearest scene cut boundary before changing the bitrate will cause the buffer to drop below a threshold. When the buffer is calculated to not drop below the threshold, the method may postpone the upcoming bitrate change until the identified nearest scene cut boundary.

Claims

1. A method comprising:

during rendering of a first coded representation of a streamed media asset, when a bitrate change is determined to occur, reviewing manifest information describing different coded representations of the media asset and having redundant content, the different coded representations of the media asset having respective bitrates, segments of the coded representations identified in the manifest information by network locations from which the segments can be retrieved, the manifest information identifying scene cut tags applying in common to temporally-coincident segments of the coded representations;

identifying a nearest scene cut boundary from scene cut tags included in the manifest information of the media asset;

calculating whether waiting until the identified scene cut boundary before changing the bitrate will cause a buffer at a bit stream sink to cross a threshold;

when the buffer is calculated to not cross the threshold, retrieving segment(s) of the first coded representation from a current playback location up to the nearest scene cut boundary from the segment(s)'"'"' network location(s) and, thereafter, retrieving segments of a second coded representation of the media asset following the nearest scene cut boundary; and

when the buffer is calculated to cross the threshold, retrieving segment(s) of the second coded representation from the current playback location up to before the nearest scene cut boundary.

2. The method of claim 1, wherein the retrieving the segments of the second coded representation before the nearest scene cut boundary includes retrieving segments of a third coded representation with a bitrate that is between a bitrate of the first coded representation and a bitrate of the second coded representation.

3. The method of claim 1, wherein when the buffer is calculated to cross the threshold, retrieving segments of a third coded representation from the current playback location up to before the retrieving the segments of the second coded representation, the third coded representation having a bitrate lower than the bitrate of the determined upcoming bitrate change.

4. The method of claim 1, wherein the upcoming bitrate change is determined based on a change in network conditions.

5. The method of claim 1, wherein the upcoming bitrate change is determined based on a change in buffer fullness.

6. The method of claim 1, wherein the upcoming bitrate change is determined based on a change in buffer fullness and a change in a network bandwidth.

7. The method of claim 1, wherein the determining the bitrate change is performed in response to a request from the bit stream sink.

8. The method of claim 1, wherein the method is performed by a distribution device that stores the first coded representation and the second coded representation.

9. The method of claim 1, wherein the method is performed by a player device.

17. The method of claim 7, further comprising:

identifying at least one intermediate coded representation of the media asset having a bitrate between the bitrates of the first coded representation and the second coded representation; and

incrementally transitioning from the bitrate of the first coded representation to the bitrate of the second coded representation through the at least one intermediate coded representation.

18. The method of claim 17, wherein the transition(s) through the intermediate coded representation(s) is made before a scene cut boundary.

19. The method of claim 17, wherein the transition(s) through the intermediate coded representation(s) is made before the nearest scene cut boundary.

20. The method of claim 1, wherein when the buffer is calculated to cross the threshold, the bitrate of the second coded representation is calculated to be the minimum bitrate necessary for the second coded representation to reach the identified scene cut boundary.

10. A system comprising:

a buffer at a bit stream sink storing a first coded representation of a streamed media asset; and

a processor to;

during rendering of the first coded representation, when a bitrate change is determined to occur, review manifest information describing different coded representations of the media asset and having redundant content, the different coded representations of the media asset having respective bitrates, segments of the coded representations identified in the manifest information by network locations from which the segments can be retrieved, the manifest information identifying scene cut tags applying in common to temporally-coincident segments of the coded representations;

identify a nearest scene cut boundary from scene cut tags included in the manifest information of the media asset;

calculate whether waiting until the identified scene cut boundary before changing the bitrate will cause the buffer to cross a threshold;

when the buffer is calculated to not cross the threshold, retrieve segment(s) of the first coded representation from a current playback location up to the nearest scene cut boundary from the segment(s)′

network location(s) and, thereafter, retrieve segments of a second coded representation of the media asset following the nearest scene cut boundary; and

when the buffer is calculated to cross the threshold, retrieve segment(s) of the second coded representation from the current playback location up to before the nearest scene cut boundary.

11. The system of claim 10, wherein the processor is on a distribution device that stores the first coded representation and the second coded representation.

12. The system of claim 10, wherein the processor is on a player device.

13. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:

during rendering of a first coded representation of a streamed media asset, when a bitrate change is determined to occur, review manifest information describing different coded representations of the media asset and having redundant content, the different coded representations of the media asset having respective bitrates, segments of the coded representations identified in the manifest information by network locations from which the segments can be retrieved, the manifest information identifying scene cut tags applying in common to temporally-coincident segments of the coded representations;

identify a nearest scene cut boundary from scene cut tags included in the manifest information of the media asset;

calculate whether waiting until the identified scene cut boundary before changing the bitrate will cause a buffer at a bit stream sink to cross a threshold;

when the buffer is calculated to not cross the threshold, retrieve segment(s) of the first coded representation from a current playback location up to the nearest scene cut boundary from the segment(s)′

network location(s) and, thereafter, retrieve segments of a second coded representation of the media asset following the nearest scene cut boundary; and

when the buffer is calculated to cross the threshold, retrieve segment(s) of the second coded representation from the current playback location up to before the nearest scene cut boundary.

14. The medium of claim 13, wherein the determining the bitrate change is performed in response to a request from the bit stream sink.

15. The medium of claim 13, wherein the processor is on a distribution device that stores the first coded representation and the second coded representation.

16. The medium of claim 13, wherein the processor is on a player device.

21. A method comprising:

during rendering of a first coded representation of a streamed media asset, when a bitrate change is determined to occur, reviewing manifest information describing different coded representations of the media asset and having redundant content, the different coded representations of the media asset having respective bitrates, segments of the coded representations identified in the manifest information by network locations from which the segments can be retrieved, the manifest information identifying scene cut tags applying in common to temporally-coincident segments of the coded representations;

identifying a nearest scene cut from scene cut tags included in the manifest information of the media asset, wherein the nearest scene cut is defined by a start frame and a subsequent end frame;

calculating whether the first coded representation will reach at least the start frame of the nearest scene cut before causing a buffer at a bit stream sink to cross a threshold;

when the buffer is calculated to not cross the threshold, retrieving segment(s) of the first coded representation from a current playback location up to the start frame of the nearest scene cut and, thereafter, retrieving segments of a second coded representation of the media asset following the start frame of the nearest scene cut; and

when the buffer is calculated to cross the threshold, retrieving segment(s) of the second coded representation from the current playback location up to before the start frame of the nearest scene cut.

22. The method of claim 21, wherein when the buffer is calculated to not cross the threshold, retrieving segments of the first coded representation from a current playback location up to a playback location between the start frame and the end frame.