Bitrate optimization for multi-representation encoding using playback statistics

• US 10,454,987 B2
• Filed: 10/28/2016
• Issued: 10/22/2019
• Est. Priority Date: 10/28/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Chen, Chao, Lin, Yao-Chung, Kokaram, Anil, Benting, Steve
• Primary Examiner(s): Sison, June Y
• Assistant Examiner(s): Cooney, Adam A
• Application Number:
  US15/337,806
• Publication Number:
  US 20180124146A1
• Time in Patent Office: 1,089 Days
• Field of Search: 709219
• CPC Class Codes: H04L 47/38 by adapting coding or compression rate
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/258 Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30

Abstract

Implementations disclose bitrate optimization for multi-representation encoding using playback statistics. A method includes generating multiple versions of a segment of a source video, the versions comprising encodings of the segment at different encoding bitrates for each resolution of the segment, measuring a quality metric for each version of the segment, generating rate-quality models for each resolution of the segment based on the measured quality metrics corresponding to the resolutions, generating a probability model to predict requesting probabilities that representations of the segment are requested, the probability model based on a joint probability distribution of network speed and viewport size that is generated from client-side feedback statistics associated with prior playbacks of other videos, determining an encoding bitrate for each of the representations of the segment based on the rate-quality models and the probability model, and assigning determined encoding bitrates to corresponding representations of the segment.

Claims

1. A method comprising:

generating multiple versions of a segment of a source video, the versions comprising encodings of the segment at different encoding bitrates for each resolution of the segment;

measuring a quality metric for each version of the segment;

generating rate-quality models for each resolution of the segment based on the measured quality metrics corresponding to the resolutions;

obtaining client-side feedback statistics associated with prior playbacks of other videos different from the source video, wherein the client-side feedback statistics comprise playback traces transmitted from media players at client devices in a same geographic region of the source video, the playback traces comprising a respective network speed and a respective viewport size utilized on a respective client device for each of the prior playbacks;

generating, by a processing device, a probability model specific to the geographic region to predict requesting probabilities that representations of the segment are requested, the probability model based on a joint probability distribution of network speed and viewport size that is generated from the client-side feedback statistics associated with the prior playbacks of the other videos;

determining, by the processing device, an encoding bitrate for each of the representations of the segment based on the rate-quality models and the probability model; and

assigning determined encoding bitrates to corresponding representations of the segment.

2. The method of claim 1, wherein the segment comprises the entire source video.

3. The method of claim 1, wherein the requesting probability for one of the representations is further based on:

the encoding bitrate of the representation and a relation of the encoding bitrate to network speed in the joint probability distribution; and

the resolution of the representation and a relation of the resolution to viewport size in the joint probability distribution.

4. The method of claim 1, wherein the joint probability distribution is generated from cumulative measurements of the network speeds determined from the playback traces and from cumulative measurements of the viewport sizes determined from the playback traces.

5. The method of claim 4, wherein the playback traces are collected for a type of the source video, and wherein the joint probability distribution is specific to the type of the source video.

6. The method of claim 1, wherein determining the encoding bitrate for each of the representations further comprises minimizing an average egress traffic for the segment such that an average quality of the segment is maintained at or above a defined quality level, wherein the average egress traffic is a function of the different encoding bitrates and the requesting probabilities, and wherein the average quality is a function of the quality metrics and the requesting probabilities.

7. The method of claim 1, wherein determining the encoding bitrate for each of the representations further comprises maximizing an average quality for the segment such that an average egress traffic of the segment is maintained at or below a defined video egress traffic level, wherein the average quality is a function of the quality metrics and the requesting probabilities, and wherein the average egress traffic is a function of the multiple bitrates and the requesting probabilities.

8. The method of claim 1, wherein assigning the determined encoding bitrates to the corresponding representations further comprises providing the selected encoding bitrates to at least one transcoder for encoding of each of the representations of the segment at the corresponding bitrate.

9. The method of claim 1, wherein the representation comprises a bitrate/resolution combination of the segment, and wherein the segment comprises one or more representations for each of the resolutions of the segment.

10. The method of claim 1, wherein the quality metric comprises at least one of a Peak Signal-to-Noise Ratio (PSNR) measurement or a Structural Similarity (SSIM) measurement.

11. A system comprising:

a memory; and

a processing device coupled to the memory, wherein the processing device is to;

obtain client-side feedback statistics associated with prior playbacks of multiple videos different from a source video, wherein the client-side feedback statistics comprise playback traces transmitted from media players at client devices in a same geographic region of the source video, the playback traces comprising a respective network speed and a respective viewport size utilized on a respective client device for each of the prior playbacks;

determine a joint probability distribution for network speed and viewport size based on the client-side feedback statistics received from client systems, the joint probability distribution specific to the geographic region;

generate rate-quality models for resolutions of a segment of the source video based on quality metrics measured for the segment;

estimate a delivered quality and egress for representations of the segment based on the generated rate-quality models and based on requesting probabilities that the representations are requested, wherein the requesting probabilities are based on the joint probability distribution; and

determine a set of bitrates comprising a bitrate to correspond to each of the representations of the segment, the set of bitrates determined to minimize the egress while maintaining the delivered quality at or above a quality threshold value.

12. The system of claim 11, wherein the requesting probability that one of the representations is requested is further based on:

the bitrate of the representation and a relation of the bitrate to network speed in the joint probability distribution; and

the resolution of the representation and a relation of the resolution to viewport size in the joint probability distribution.

13. The system of claim 11, wherein the joint probability distribution is generated from cumulative measurements of the network speeds determined from the playback traces and from cumulative measurements of the viewport sizes determined from the playback traces.

14. The system of claim 13, wherein the playback traces are collected for a type of the source video, and wherein the joint probability distribution is specific to the type of the source video.

15. The system of claim 11, wherein the processing device is further to provide the determined set of bitrates to at least one transcoder for encoding of each of the representations of the segment at the corresponding bitrate.

16. The system of claim 11, wherein the delivered quality is based on at least one of Peak Signal-to-Noise Ratio (PSNR) measurements of the encodings or Structural Similarity (SSIM) measurements of the encodings.

17. A non-transitory machine-readable storage medium storing instructions which, when executed, cause a processing device to perform operations comprising:

obtaining client-side feedback statistics associated with prior playbacks of multiple videos different from a source video, wherein the client-side feedback statistics comprise playback traces transmitted from media players at client devices in a same geographic region of the source video, the playback traces comprising a respective network speed and a respective viewport size utilized on a respective client device for each of the prior playbacks;

determining a joint probability distribution for network speed and viewport size based on the client-side feedback statistics received from client systems, the joint probability distribution specific to the geographic region;

generating rate-quality models for resolutions of a segment of a video based on quality metrics measure for the segment;

estimating, by the processing device, a delivered quality and egress for representations of the segment based on the generated rate-quality models and based on requesting probabilities that the representations is requested, wherein the requesting probabilities are based on the joint probability distribution; and

determining, by the processing device, a set of bitrates comprising a bitrate to correspond to each of the representations of the segment, the set of bitrates determined to maximize the delivered quality while keeping the egress at or below an egress threshold value.

18. The non-transitory machine-readable storage medium of claim 17, wherein the requesting probability that one of the representations is requested is further based on:

the bitrate of the representation and a relation of the bitrate to network speed in the joint probability distribution; and

the resolution of the representation and a relation of the resolution to viewport size in the joint probability distribution.

19. The non-transitory machine-readable storage medium of claim 17, wherein the joint probability distribution is generated from cumulative measurements of the network speeds determined from the playback traces and from cumulative measurements of the viewport sizes determined from the playback traces.

20. The non-transitory machine-readable storage medium of claim 19, wherein the playback traces are collected for a type of the source video, and wherein the joint probability distribution is specific to the type of source video.

21. The non-transitory machine-readable storage medium of claim 17, wherein the processing device is further to provide the determined set of bitrates to at least one transcoder for encoding of each of the representations of the segment at the corresponding bitrate.

22. The non-transitory machine-readable storage medium of claim 17, wherein the delivered quality is based on at least one of Peak Signal-to-Noise Ratio (PSNR) measurements of the segment or Structural Similarity (SSIM) measurements of the segment.