Method and system for parallel encoding of a video

• US 9,247,264 B2
• Filed: 12/01/2011
• Issued: 01/26/2016
• Est. Priority Date: 12/01/2010
• Status: Active Grant

• Current Assignee: Ecole de technologie superieure
• Original Assignee: Ecole de technologie superieure
• Inventors: Coulombe, Stephane, Franche, Jean-Francois
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): PEREZ FUENTES, LUIS M
• Application Number:
  US13/309,298
• Publication Number:
  US 20120140816A1
• Time in Patent Office: 1,517 Days
• Field of Search: 375/240.03, 375/240.13
• US Class Current: 1/1
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A method and system for parallel encoding of frames in a video are described, exploiting parallel processing at both frame and slice levels. One or more slices are processed in a preceding frame by one or more cores in a multi-core processor, and a slice is processed in a succeeding frame provided at least one of said one or more cores in the multi-core processor is available for processing; and the processing of a slice from said one or more slices at same physical location in the preceding input frame has been completed, the processing of frames and slices in the preceding and succeeding frames being performed at least partially in parallel. A significant speedup in comparison to the sequential encoding approach is achieved while maintaining high visual quality for the output video. A corresponding system for parallel encoding of a video is also provided.

Claims

1. A method for encoding frames in a video, comprising:

employing at least one hardware processor for;

encoding two successive input frames of the video, a preceding input frame and a succeeding input frame, comprising;

(a1) dividing the preceding input frame into slices, each slice characterizing a spatial region within the preceding input frame;

(a2) dividing the succeeding input frame into slices, each slice characterizing a spatial region within the succeeding input frame;

and in parallel;

(a3) processing one or more slices in the preceding input frame by a multi-core processor;

(a4) after the processing of one of said one or more slices in the preceding input frame has been completed, processing, by those cores of the multi-core processor that are available for processing, one or more slices in the succeeding input frame having locations corresponding to the one or more slices in the preceding input frame, comprising;

for a slice of said one or more slices in the succeeding input frame for which a corresponding slice in the preceding input frame has been already processed;

(i) performing motion estimation using all slices in a motion estimation window for the corresponding slice in the preceding input frame, provided said all slices in the motion estimation window in the preceding input frame have been processed;

(ii) provided said all slices in the motion estimation window for the corresponding slice in the preceding input frame have not been processed, performing the motion estimation using only those slices in the motion estimation window in the preceding input frame that have been already processed; and

(iii) provided none of the slices in the motion estimation window in the preceding input frame have been processed except for the corresponding slice, performing the motion estimation using the corresponding slice in the preceding input frame.

2. The method of claim 1, wherein the preceding input frame and the succeeding input frame are intra frames.

3. The method of claim 1, wherein the preceding input frame and the succeeding input frame are predicted frames.

4. The method of claim 1, further comprising performing the step (a4) immediately after the processing of at least one of the slices in the preceding input frame has been completed.

5. The method of claim 1, wherein the step (ii) further comprises performing the motion estimation using:

the corresponding slice in the preceding input frame; and

one of the following;

at least one slice on top or at bottom of the corresponding slice in the preceding input frame;

both a slice on top and a slice at bottom of the corresponding slice in the preceding input frame;

more than one slice above the corresponding slice in the preceding input frame;

more than one slice below the corresponding slice in the preceding input frame.

6. The method of claim 1, further comprising:

for each input frame, generating a corresponding output frame comprising processed slices; and

writing the output frames in an output video.

7. The method of claim 1, wherein a sum of a number of slices in the preceding input frame and a number of slices in the succeeding input frame is equal to or greater than a number of cores in the multi-core processor.

8. The method of claim 1, wherein:

the step (a3) further comprises performing a pre-slice encoding of the preceding input frame before processing said one or more slices in the preceding input frame, comprising writing a frame type and a frame header into a respective output frame;

the step (a4) further comprises performing a pre-slice encoding of the succeeding input frame before processing the slice in the succeeding input frame, comprising writing a frame type and a frame header into a respective output frame; and

the pre-slice encoding of the succeeding input frame being performed after the pre-slice encoding of the preceding input frame is completed.

9. The method of claim 1, wherein:

the step (a3) further comprises performing a post-slice encoding of the preceding input frame after processing all slices in the preceding input frame, comprising applying a deblocking filter for smoothing sharp edges formed between macroblocks of the preceding input frame;

the step (a4) further comprises performing a post-slice encoding of the succeeding input frame after processing all slices in the succeeding input frame, comprising applying a deblocking filter for smoothing sharp edges formed between macroblocks of the succeeding input frame; and

the post-slice encoding of the succeeding input frame being performed after the post-slice encoding of the preceding input frame is completed.

10. The method of claim 1, wherein a sum of a number of slices in the preceding input frame and a number of slices in the succeeding input frame is about one and a half times a number of cores in the multi-core processor.

11. The method of claim 1, wherein:

the step (a3) further comprises performing quantization for the preceding input frame producing a quantization parameter; and

the step (a4) further comprises performing quantization for the succeeding input frame by using the quantization parameter.

12. The method of claim 6, wherein the output video is in a H.264 format.

13. A non-transitory computer readable storage medium, having a computer readable program code instructions stored thereon for execution by a hardware processor, causing the hardware processor to:

encode two successive input frames of the video, a preceding input frame and a succeeding input frame, comprising;

(a1) dividing the preceding input frame into slices, each slice characterizing a spatial region within the preceding input frame;

(a2) dividing the succeeding input frame into slices, each slice characterizing a spatial region within the succeeding input frame;

and in parallel;

(a3) processing one or more slices in the preceding input frame by a multi-core processor; and

after the processing of one of the one or more slices in the preceding input frame has been completed;

(a4) processing, by those cores of the multi-core processor that are available for processing, one or more slices in the succeeding input frame having locations corresponding to the one or more slices in the preceding input frame, comprising;

for a slice of said one or more slices in the succeeding input frame for which a corresponding slice in the preceding input frame has been already processed;

(i) performing motion estimation using all slices in a motion estimation window for the corresponding slice in the preceding input frame, provided said all slices in the motion estimation window in the preceding input frame have been processed;

(ii) provided said all slices in the motion estimation window for the corresponding slice in the preceding input frame have not been processed, performing the motion estimation using only those slices in the motion estimation window in the preceding input frame that have been already processed; and

(iii) provided none of the slices in the motion estimation window in the preceding input frame have been processed except for the corresponding slice, performing the motion estimation using the corresponding slice in the preceding input frame.

14. A computerized system for encoding frames in a video comprising:

a hardware processor, and a non-transitory computer readable storage medium having computer readable instructions stored thereon for execution by the processor, causing the hardware processor to;

encode two successive input frames of the video, a preceding input frame and a succeeding input frame, comprising;

(a1) dividing the preceding input frame into slices characterizing respective spatial regions within the preceding input frame, and the succeeding input frame into slices characterizing respective spatial regions within the succeeding input frame;

in parallel;

(a2) processing one or more slices in the preceding input frame by a multi-core processor; and

after the processing of one of the one or more slices in the preceding input frame has been completed;

(a3) processing, by those cores of the multi-core processor that are available for processing, one or more slices in the succeeding input frame having locations corresponding to the one or more slices in the preceding input frame, comprising;

for a slice of said one or more slices in the succeeding input frame for which a corresponding slice in the preceding input frame has been already processed;

(i) performing motion estimation using all slices in a motion estimation window for the corresponding slice in the preceding input frame, provided said all slices in the motion estimation window in the preceding input frame have been processed;

(ii) provided said all slices in the motion estimation window for the corresponding slice in the preceding input frame have not been processed, performing the motion estimation using only those slices in the motion estimation window in the preceding input frame that have been already processed; and

(iii) provided none of the slices in the motion estimation window in the preceding input frame have been processed except for the corresponding slice, performing the motion estimation using the corresponding slice in the preceding input frame.

15. The system of claim 14, wherein the preceding input frame and the succeeding input frame are intra frames.

16. The system of claim 14, wherein the preceding input frame and the succeeding input frame are predicted frames.

17. The system of claim 14, wherein the computer readable instructions further cause the processor to perform the processing of a slice in the succeeding input frame immediately after the processing of at least one of the one or more slices in the preceding input frame has been completed.

18. The system of claim 14, wherein the computer readable instructions further cause the processor to perform the motion estimation using:

the corresponding slice in the preceding input frame; and

one of the following;

at least one slice on top or at bottom of the corresponding slice in the preceding input frame;

both a slice on top and a slice at bottom of the corresponding slice in the preceding input frame;

more than one slice above the corresponding slice in the preceding input frame;

more than one slice below the corresponding slice in the preceding input frame.

19. The system of claim 14, wherein the computer readable instructions further cause the processor to generate:

for each input frame, a corresponding output frame comprising processed slices; and

a writer module writing the output frames in an output video.

20. The system of claim 14, wherein a sum of a number of slices in the preceding input frame and a number of slices in the succeeding input frame is equal to or greater than a number cores in the multi-core processor.

21. The system of claim 14, wherein the computer readable instructions further cause the processor to:

perform a pre-slice encoding of the preceding input frame before processing said one or more slices in the preceding input frame, comprising writing a frame type and a frame header into a respective output frame; and

perform a pre-slice encoding of the succeeding input frame before processing the slice in the succeeding input frame, comprising writing a frame type and a frame header into a respective output frame;

wherein the pre-slice encoding of the succeeding input frame is performed after the pre-slice encoding of the preceding input frame is completed.

22. The system of claim 14, wherein the computer readable instructions further cause the processor to:

perform a post-slice encoding of the preceding input frame after processing all slices in the preceding input frame, comprising applying a deblocking filter for smoothing sharp edges formed between macroblocks of the preceding input frame; and

perform a post-slice encoding of the succeeding input frame after processing all slices in the succeeding input frame, comprising applying a deblocking filter for smoothing sharp edges formed between macroblocks of the succeeding input frame;

wherein the post-slice encoding of the succeeding input frame is performed after the post-slice encoding of the preceding input frame is completed.

23. The system of claim 14, wherein a sum of a number of slices in the preceding input frame and a number of slices in the succeeding input frame is about one and a half times a number of cores in the multi-core processor.

24. The system of claim 14, wherein the computer readable instructions further cause the processor to:

perform quantization for the preceding input frame to produce a quantization parameter; and

perform quantization for the succeeding input frame by using the quantization parameter.

25. The system of claim 19, wherein the output video is in a H.264 format.