Segment based encoding of video

• US 10,063,872 B2
• Filed: 09/11/2015
• Issued: 08/28/2018
• Est. Priority Date: 09/11/2015
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Coward, Michael Hamilton, Puntambekar, Amit, George, Thomas
• Primary Examiner(s): Beck, Leron
• Application Number:
  US14/852,474
• Publication Number:
  US 20170078687A1
• Time in Patent Office: 1,082 Days
• Field of Search: 37524001-24029
• CPC Class Codes: H04N 19/395 involving distributed video coding [DVC], e.g. Wyner-Ziv video coding or Slepian-Wolf video coding
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

A distributed video encoding system splits an input video into video segments. The video segments are encoded using multiple video encoding nodes. Prior to the process of splitting the sequence into video segments, the video is analyzed to generate a dependency map. Intelligent segmentation is performed using the dependency map so that each video segment includes all the video frames from which other video frames within that segment have been encoded in the input video. For example, picture headers are inspected to determine the temporal distance of the farthest past and future reference frames used for encoding frames of a video.

Claims

1. A computer-implemented method for encoding a compressed video bitstream, comprising:

receiving a compressed video bitstream in which key frames occur nominally a first time duration apart, wherein a key frame is encoded without depending on another frame in the compressed video bitstream;

building a dependency map for the compressed video bitstream, wherein the dependency map identifies the key frames;

segmenting, using the dependency map, the compressed video bitstream in segments, wherein at least some segments have overlapping frames;

providing segments of the compressed video bitstream corresponding to a nominal time duration to multiple encoding nodes, wherein each segment includes at least one key frame, and wherein at least some segments include video frames that occur temporally before or after the at least one key frame in a display order;

indicating, to the multiple encoding nodes, a number of frames to skip from encoding at beginning of each segment and a number of frames to skip encoding at end of each segment, such that at least some segments include frames that are not to be encoded by a corresponding encoding node; and

encoding, by each encoding nodes, the corresponding segment of compressed video bitstream such that key frames in resulting encoded video bitstream occur a second time duration apart.

2. The computer-implemented method of claim 1, wherein the encoding includes encoding by changing temporal location of a key frame in resulting encoded video bitstream.

3. The computer-implemented method of claim 1, further including determining, for each frame, in the display order, most distant past and future frames from which motion vectors for each frame are formed.

4. The computer-implemented method of claim 3, wherein the determining is performed using motion vector information from the compressed video bitstream.

5. The computer-implemented method of claim 3, wherein the determining is performed without uncompressing the compressed video bitstream.

6. The computer-implemented method of claim 1, wherein the encoding by each node is performed on fewer than all frames making up the corresponding segment.

7. The computer-implemented method of claim 1, further comprising:

providing, to each of the multiple encoding nodes, information regarding location of a key frame in the encoding operation.

8. The computer-implemented method of claim 7, further including encoding, by one of the multiple encoding nodes, the at least one key frame in received video segment to become a non-key frame in resulting encoded video bitstream.

9. A non-transitory computer-readable medium storing instructions that, upon execution, cause at least one computer processor to perform a method, the instructions comprising instructions for:

instructions for receiving a compressed video bitstream in which key frames occur nominally a first time duration apart, wherein a key frame is encoded without depending on another frame in the compressed video bitstream;

instructions for building a dependency map for the compressed video bitstream, wherein the dependency map identifies the key frames;

instructions for segmenting, using the dependency map, the compressed video bitstream in segments, wherein at least some segments have overlapping frames;

instructions for providing segments of the compressed video bitstream corresponding to a nominal time duration to multiple encoding nodes, wherein each segment includes at least one key frame, and wherein at least some segments include video frames that occur temporally before or after the at least one key frame in a display order, and wherein at least some segments include frames that are not to be encoded by a corresponding encoding node;

instructions for indicating, to the multiple encoding nodes, a number of frames to skip from encoding at beginning of each segment and a number of frames to skip encoding at end of each segment, such that at least some segments include frames that are not to be encoded by a corresponding encoding node;

instructions for encoding, by each encoding nodes, the corresponding segment of compressed video bitstream such that key frames in resulting encoded video bitstream occur a second time duration apart.

10. The non-transitory computer-readable medium of claim 9, wherein the instructions for encoding include instructions for encoding by changing temporal location of a key frame in resulting encoded video bitstream.

11. The non-transitory computer-readable medium of claim 9, wherein the instructions further include instructions for determining, for each frame, in the display order, most distant past and future frames from which motion vectors for each frame are formed.

12. The non-transitory computer-readable medium of claim 11, wherein the instructions for determining include instructions for using motion vector information from the compressed video bitstream.

13. The non-transitory computer-readable medium of claim 11, wherein the instructions for determining include instructions for determining without uncompressing the compressed video bitstream.

14. The non-transitory computer-readable medium of claim 9, wherein the encoding by each node is performed on fewer than all frames making up the corresponding segment.

15. A computer system, comprising:

at least one processor;

at least one memory comprising instructions configured to cause the at least one processor to perform a method, comprising;

receiving a compressed video bitstream in which key frames occur nominally a first time duration apart, wherein a key frame is encoded without depending on another frame in the compressed video bitstream;

building a dependency map for the compressed video bitstream, wherein the dependency map identifies the key frames;

segmenting, using the dependency map, the compressed video bitstream in segments, wherein at least some segments have overlapping frames;

providing segments of the compressed video bitstream corresponding to a nominal time duration to multiple encoding nodes, wherein each segment includes at least one key frame, and wherein at least some segments include video frames that occur temporally before or after the at least one key frame in a display order, and wherein at least some segments include frames that are not to be encoded by a corresponding encoding node;

indicating, to the multiple encoding nodes, a number of frames to skip from encoding at beginning of each segment and a number of frames to skip encoding at end of each segment, such that at least some segments include frames that are not to be encoded by a corresponding encoding node; and

encoding, by each encoding nodes, the corresponding segment of compressed video bitstream such that key frames in resulting encoded video bitstream occur a second time duration apart.

16. The computer system of claim 15, wherein the encoding includes encoding by changing temporal location of a key frame in resulting encoded video bitstream.

17. The computer system of claim 15, wherein the method further includes determining, for each frame, in the display order, most distant past and future frames from which motion vectors for each frame are formed.

18. The computer system of claim 17, wherein the determining includes determining using motion vector information from the compressed video bitstream.

19. The computer system of claim 17, wherein the determining includes determining without uncompressing the compressed video bitstream.

20. The computer system of claim 15, wherein the encoding by each node is performed on fewer than all frames making up the corresponding segment.