Variable bit video streams for adaptive streaming

• US 9,930,385 B2
• Filed: 03/26/2015
• Issued: 03/27/2018
• Est. Priority Date: 12/06/2010
• Status: Active Grant

• Current Assignee: Netflix, Inc.
• Original Assignee: Netflix, Inc.
• Inventors: Hunt, Neil D.
• Primary Examiner(s): HANCE, ROBERT J
• Application Number:
  US14/670,194
• Publication Number:
  US 20150201226A1
• Time in Patent Office: 1,097 Days
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/23655 Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
• CPC Class Codes: H04N 21/238 Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams hybrid fiber coaxial [HFC] networks for downstream channel allocation for video distribution H04L12/2801; flow control in packet networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/64769 for rate control flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 5/76 Television signal recording

Abstract

One embodiment of the present invention sets forth a technique for adapting playback bit rate in a content delivery system based on scene complexity of the video content as well as network conditions and other performance factors. A scene complexity map of the video content indicates the complexity levels of different scenes within the video content. Using the scene complexity map, a content player may download scenes of lower scene complexity levels from video streams encoded to lower bit rates to manage the bandwidth consumed to download the video content and allow for the downloading of scenes of higher complexity levels from video streams encoded to higher bit rates.

Claims

1. A method, comprising:

computing a first estimated bandwidth available for downloading digital content from one or more content servers during a first time window and a second estimated available bandwidth available for downloading the digital content from the one or more content servers during a second time window based on actual bandwidths that were available for downloading the digital content from the one or more content servers during one or more previous time windows;

computing a bandwidth variability based on the actual bandwidths available during the one or more previous time windows and estimated bandwidths computed for the one or more previous time windows;

determining from a scene complexity map a first complexity level for the digital content within the first time window and a second complexity level for the digital content within the second time window; and

selecting a first encoded portion of the digital content to download for playback during the first time window from a first content stream included in a plurality of encoded content streams,

wherein each encoded content stream comprises the digital content encoded at a different bit rate, and the first content stream is encoded at a first bit rate, and

wherein the first encoded portion is selected based on the first estimated bandwidth available during the first time window, the second estimated bandwidth available during the second time window, the bandwidth variability, and a desired bit rate at which a second encoded portion of the digital content to be downloaded subsequent to the first encoded portion is encoded.

2. The method of claim 1, wherein the first estimated bandwidth available during the first time window comprises a high-level bandwidth value, the first complexity level comprises a low-level complexity, the second estimated bandwidth available during the second window comprises a mid-level bandwidth value, the second complexity level comprises a high-level complexity, the desired bit rate at which the second encoded portion is encoded comprises a high-level bit rate, and the first encoded content stream from which first encoded portion is selected is encoded at a low-level bit rate.

3. The method of claim 2, further comprising downloading the first encoded portion during the first time window, selecting the second encoded portion of the digital content from a second encoded content stream included in the plurality of encoded content streams that is encoded at a high-level bit rate, and downloading at least a portion of the second encoded portion during the first time window.

4. The method of claim 1, wherein the first estimated bandwidth available during the first time window comprises a mid-level bandwidth value, the first complexity level comprises a mid-level complexity, the second estimated bandwidth available during the second window comprises a mid-level bandwidth value, the second complexity level comprises a high-level complexity, the desired bit rate at which the second encoded portion is encoded comprises a high-level bit rate, and the first encoded content stream from which first encoded portion is selected is encoded at a low-level bit rate.

5. The method of claim 4, further comprising downloading at least a portion of the first encoded portion during a time window that immediately precedes the first time window, selecting the second encoded portion of the digital content from a second encoded content stream included in the plurality of encoded content streams that is encoded at a high-level bit rate, and downloading at least a portion of the second encoded portion during the first time window.

6. The method of claim 1, wherein the first estimated bandwidth available during the first time window comprises a mid-level bandwidth value, the first complexity level comprises a mid-level complexity, the second estimated bandwidth available during the second window comprises a high-level bandwidth value, the second complexity level comprises a low-level complexity, and the first encoded content stream from which first encoded portion is selected is encoded at a mid-level bit rate.

7. The method of claim 6, wherein the desired bit rate at which the second encoded portion is encoded comprises either high-level bit rate, a mid-level bit rate, or a low-level bit rate, and further comprising downloading the first encoded portion during the first time window, selecting the second encoded portion of the digital content from a second encoded content stream included in the plurality of encoded content streams that is encoded at the desired bit rate, and downloading the second encoded portion during the second time window.

8. The method of claim 1, wherein the digital content comprises digital video content associated with a movie title.

9. A non-transitory computer-readable medium including instructions that, when executed by a processor, cause the processor to download digital content from one or more content servers, by performing the steps of:

computing a first estimated bandwidth available for downloading digital content from the one or more content servers during a first time window and a second estimated available bandwidth available for downloading the digital content from the one or more content servers during a second time window based on actual bandwidths that were available for downloading the digital content from the one or more content servers during one or more previous time windows;

determining from a scene complexity map a first complexity level for the digital content within the first time window and a second complexity level for the digital content within the second time window; and

selecting a first encoded portion of the digital content to download for playback during the first time window from a first content stream included in a plurality of encoded content streams,

wherein each encoded content stream comprises the digital content encoded at a different bit rate, and the first content stream is encoded at a first bit rate, and

wherein the first encoded portion is selected based on the first estimated bandwidth available during the first time window, the second estimated bandwidth available during the second time window, and a desired bit rate at which a second encoded portion of the digital content to be downloaded subsequent to the first encoded portion is encoded.

10. The non-transitory computer-readable medium of claim 9, wherein the first estimated bandwidth available during the first time window comprises a high-level bandwidth value, the first complexity level comprises a low-level complexity, the second estimated bandwidth available during the second window comprises a mid-level bandwidth value, the second complexity level comprises a high-level complexity, the desired bit rate at which the second encoded portion is encoded comprises a high-level bit rate, and the first encoded content stream from which first encoded portion is selected is encoded at a low-level bit rate.

11. The non-transitory computer-readable medium of claim 10, further comprising downloading the first encoded portion during the first time window, selecting the second encoded portion of the digital content from a second encoded content stream included in the plurality of encoded content streams that is encoded at a high-level bit rate, and downloading at least a portion of the second encoded portion during the first time window.

12. The non-transitory computer-readable medium of claim 9, wherein the first estimated bandwidth available during the first time window comprises a mid-level bandwidth value, the first complexity level comprises a mid-level complexity, the second estimated bandwidth available during the second window comprises a mid-level bandwidth value, the second complexity level comprises a high-level complexity, the desired bit rate at which the second encoded portion is encoded comprises a high-level bit rate, and the first encoded content stream from which first encoded portion is selected is encoded at a low-level bit rate.

13. The non-transitory computer-readable medium of claim 12, further comprising downloading at least a portion of the first encoded portion during a time window that immediately precedes the first time window, selecting the second encoded portion of the digital content from a second encoded content stream included in the plurality of encoded content streams that is encoded at a high-level bit rate, and downloading at least a portion of the second encoded portion during the first time window.

14. The non-transitory computer-readable medium of claim 9, wherein the first estimated bandwidth available during the first time window comprises a mid-level bandwidth value, the first complexity level comprises a mid-level complexity, the second estimated bandwidth available during the second window comprises a high-level bandwidth value, the second complexity level comprises a low-level complexity, and the first encoded content stream from which first encoded portion is selected is encoded at a mid-level bit rate.

15. The non-transitory computer-readable medium of claim 14, wherein the desired bit rate at which the second encoded portion is encoded comprises either high-level bit rate, a mid-level bit rate, or a low-level bit rate, and further comprising downloading the first encoded portion during the first time window, selecting the second encoded portion of the digital content from a second encoded content stream included in the plurality of encoded content streams that is encoded at the desired bit rate, and downloading the second encoded portion during the second time window.

16. The non-transitory computer-readable medium of claim 9, wherein the digital content comprises digital video content associated with a movie title.

17. The non-transitory computer-readable medium of claim 9, further comprising computing a bandwidth variability based on the actual bandwidths available during the one or more previous time windows and estimated bandwidths computed for the one or more previous time windows, wherein the first encoded portion is also selected based on the bandwidth variability.

18. A computing device, comprising:

a processor; and

a memory that stores a scene complexity map and a playback application,

wherein the scene complexity map specifies complexity levels for different scenes included in digital content stored on one or more content servers, and

wherein the playback engine, when executed by the processor, is configured to;

compute a first estimated bandwidth available for downloading the digital content from the one or more content servers during a first time window and a second estimated available bandwidth available for downloading the digital content from the one or more content servers during a second time window based on actual bandwidths that were available for downloading the digital content from the one or more content servers during one or more previous time windows;

determine from the scene complexity map a first complexity level for the digital content within the first time window and a second complexity level for the digital content within the second time window; and

selecting a first encoded portion of the digital content to download for playback during the first time window from a first content stream included in a plurality of encoded content streams,

wherein each encoded content stream comprises the digital content encoded at a different bit rate, and the first content stream is encoded at a first bit rate, and

wherein the first encoded portion is selected based on the first estimated bandwidth available during the first time window, the second estimated bandwidth available during the second time window, and a desired bit rate at which a second encoded portion of the digital content to be downloaded subsequent to the first encoded portion is encoded.

19. The computing device of claim 18, wherein the scene complexity map is generated by a complexity map generator residing on one of the one or more content servers.

20. The computing device of claim 18, wherein the playback engine is further configured to compute a bandwidth variability based on the actual bandwidths available during the one or more previous time windows and estimated bandwidths computed for the one or more previous time windows, and wherein the playback engine is further configured to select the first encoded portion based on the bandwidth variability.