Systems and methods of encoding multiple video streams for adaptive bitrate streaming

• US 10,178,399 B2
• Filed: 05/16/2016
• Issued: 01/08/2019
• Est. Priority Date: 02/28/2013
• Status: Expired due to Fees

• Current Assignee: NLD Holdings I LLC
• Original Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Inventors: Orton-Jay, Sam, Naletov, Ivan Vladimirovich
• Primary Examiner(s): Ustaris, Joseph G
• Assistant Examiner(s): Adrovel, William
• Application Number:
  US15/156,126
• Publication Number:
  US 20160261879A1
• Time in Patent Office: 967 Days
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/194 involving only two passes
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements

Abstract

Systems and methods for encoding multiple video streams in accordance with embodiments of the invention are disclosed. In one embodiment, a source encoder configured to encode source video as a number of alternative video streams includes a parallel processing system configured to collect statistics on source video data and write the statistics to shared memory in a first pass through the received multimedia content, determine initial encoding information for source video data and write the initial encoding information to shared memory during the first, encode the source video data in parallel using collected statistics and initial encoding information to produce a plurality of alternative video streams during a second pass, and parallel encoding processes are configured to reuse additional encoding information that has already been determined for a portion of video and to generate additional encoding information that has not already been determined for a portion of video.

Claims

1. A source encoder configured to encode source video as a number of alternative video streams, the source encoder comprising:

memory containing a source encoder application;

a shared memory; and

a parallel processing system configured by the source encoding application to;

receive multimedia content, where the multimedia content comprises source video data having a primary resolution;

collect statistics on source video data and write the statistics to shared memory in a first pass through the received multimedia content;

determine initial encoding information for source video data and write the initial encoding information to shared memory during the first pass through the received multimedia content; and

encode the source video data in parallel using collected statistics and initial encoding information to produce a plurality of alternative video streams during a second pass through the received multimedia content with a plurality of parallel encoding processes, where the encoding of the source video utilizes additional encoding information, the parallel encoding processes are configured to reuse additional encoding information that has already been determined for a portion of video by another parallel encoding process and stored in the shared memory, and the parallel encoding processes are configured to generate additional encoding information that has not already been determined for a portion of video by another parallel encoding process and store the generated additional encoding information in the shared memory;

wherein the parallel encoding processes being configured to generate additional encoding information that has not already been determined for a portion of video by another parallel encoding process further comprises determining a Coding Tree Unit (CTU) size for encoding a portion of a frame of video in the source video data;

wherein determining a CTU size for encoding a portion of a frame of video in the source video data comprises;

selecting a portion of a frame of video in the source video data to encode as an output CTU in a first output stream;

checking if a size has been determined for a similar CTU previously encoded in a second output stream for the selected portion of a frame of video;

comparing the resolution of the first output stream to the resolution of the second output stream if a size has been determined for the similar CTU; and

scaling the CTU size of the similar CTU and applying the scaled CTU size to encoding the portion of the frame of video as the output CTU if the resolution of the first output stream is not the same resolution as the second output stream;

wherein the parallel processing system being configured to determine initial encoding information for source video data further comprises the parallel processing system configured to determine a mode distribution for at least one frame of video in at least one of the plurality of alternative video streams; and

wherein the parallel processing system being configured to encode the source video data in parallel using collected statistics and initial encoding information to produce a plurality of alternative video streams further comprises the parallel processing system being configured to;

maintain a count of blocks processed in a frame of video in an alternative video stream;

determine a threshold number of blocks based on the mode distribution; and

adjust criteria for block type decisions if the count of blocks meets the threshold number of blocks.

2. The source encoder of claim 1 wherein statistics on source video data comprises statistics selected from the group consisting of:

average quantization parameter, size of header bits, size of texture bits, number of intra blocks, number of inter blocks, and number of skip blocks.

3. The source encoder of claim 1 wherein the parallel processing system being configured to determine initial encoding information for source video data further comprises the parallel processing system configured to calculate a frame complexity measure.

4. The source encoder of claim 1 wherein the parallel encoding processes being configured to reuse additional encoding information that has already been determined for a portion of video by another parallel encoding process and stored in the shared memory further comprises the parallel encoding processes being configured to:

determine whether a motion vector exists for a second corresponding block in a second alternative stream when encoding a first block in a video frame in a first alternative stream;

determine if the first alternative stream and the second alternative stream are the same resolution;

scale the motion vector if the first alternative stream and the second alternative stream are not the same resolution;

refine the motion vector; and

apply the motion vector in encoding the first block in the video frame in the first alternative stream.

5. The source encoder of claim 1 wherein initial encoding information further comprises header size, macroblock size, and the relative proportion of header size to macroblock size.

6. The source encoder of claim 1 wherein initial encoding information further comprises hypothetical reference decoder data.

7. The source encoder of claim 1 wherein each of the parallel encoding processes encodes at a different resolution.

8. The source encoder of claim 7 wherein each of the parallel encoding processes encodes one or more alternative video streams and each of the alternative video streams encoded by a parallel encoding process is a different bitrate.

9. The source encoder of claim 1 wherein each of the parallel encoding processes encodes a block from the source video data into each stream in a subset of the plurality of alternative video streams sequentially one after another.

10. The source encoder of claim 1 wherein additional encoding information comprises rate distortion information and quantization parameters.

11. A method for encoding source video as a number of alternative video streams, the method comprising:

receiving multimedia content, where the multimedia content comprises source video data having a primary resolution using a source encoder;

collecting statistics on source video data and writing the statistics to shared memory in a first pass through the received multimedia content using a source encoder;

determining initial encoding information for source video data and writing the initial encoding information to a shared memory during the first pass through the received multimedia content using a source encoder; and

encoding the source video data in parallel using collected statistics, initial encoding information, and additional encoding information to produce a plurality of alternative video streams during a second pass through the received multimedia content with a plurality of parallel encoding processes using a source encoder, where encoding of the source video further comprises;

reusing additional encoding information that has already been determined for a portion of video by another parallel encoding process and stored in the shared memory using at least one of the plurality of parallel encoding processes;

generating additional encoding information that has not already been determined for a portion of video by another of the plurality of parallel encoding processes; and

storing the generated additional encoding information in the shared memory using a parallel encoder process;

wherein generating additional encoding information that has not already been determined for a portion of video by another parallel encoding process further comprises determining a Coding Tree Unit (CTU) size for encoding a portion of a frame of video in the source video data; and

wherein determining a CTU size for encoding a portion of a frame of video in the source video data comprises;

selecting a portion of a frame of video in the source video data to encode as an output CTU in a first output stream;

checking if a size has been determined for a similar CTU previously encoded in a second output stream for the selected portion of a frame of video;

comparing the resolution of the first output stream to the resolution of the second output stream if a size has been determined for the similar CTU; and

scaling the CTU size of the similar CTU and applying the scaled CTU size to encoding the portion of the frame of video as the output CTU if the resolution of the first output stream is not the same resolution as the second output stream;

wherein determining initial encoding information for source video data further comprises determining a mode distribution for at least one frame of video in at least one of the plurality of alternative video streams; and

wherein encoding the source video data in parallel using collected statistics, initial encoding information, and additional encoding information to produce a plurality of alternative video streams further comprises;

maintaining a count of blocks processed in a frame of video in an alternative video stream;

determining a threshold number of blocks based on the mode distribution; and

adjusting criteria for block type decisions if the count of blocks meets the threshold number of blocks.

12. The method of claim 11 wherein statistics on source video data comprises statistics selected from the group consisting of:

average quantization parameter, size of header bits, size of texture bits, number of intra blocks, number of inter blocks, and number of skip blocks.

13. The method of claim 11 wherein determining initial encoding information for source video data further comprises calculating a frame complexity measure.

14. The method of claim 11 wherein reusing additional encoding information that has already been determined for a portion of video by another parallel encoding process and stored in the shared memory further comprises:

determining whether a motion vector exists for a second corresponding block in a second alternative stream when encoding a first block in a video frame in a first alternative stream;

determining if the first alternative stream and the second alternative stream are the same resolution;

scaling the motion vector if the first alternative stream and the second alternative stream are not the same resolution;

refining the motion vector; and

applying the motion vector in encoding the first block in the video frame in the first alternative stream.

15. The method of claim 11 wherein initial encoding information further comprises header size, macroblock size, and the relative proportion of header size to macroblock size.

16. The method of claim 11 wherein initial encoding information further comprises hypothetical reference decoder data.

17. The method of claim 11 wherein each of the parallel encoding processes encodes at a different resolution.

18. The source encoder of claim 17 wherein each of the parallel encoding processes encodes one or more alternative video streams and each of the alternative video streams encoded by a parallel encoding process is a different bitrate.

19. The method of claim 11 wherein each of the parallel encoding processes encodes a block from the source video data into each stream in a subset of the plurality of alternative video streams sequentially one after another.

20. The method of claim 11 wherein additional encoding information comprises rate distortion information and quantization parameters.