METHOD AND SYSTEM FOR PARALLEL ENCODING OF A VIDEO

• US 20120140816A1
• Filed: 12/01/2011
• Published: 06/07/2012
• Est. Priority Date: 12/01/2010
• Status: Active Grant

• Current Assignee: Ecole de technologie superieure
• Original Assignee: Ecole de technologie superieure
• Inventors: Coulombe, Stephane, FRANCHE, Jean-Francois
• Granted Patent: US 9,247,264 B2
• Time in Patent Office: Days
• US Class Current: 375/240.03
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A method and system for parallel encoding of frames in a video are described, exploiting parallel processing at both frame and slice levels. One or more slices are processed in a preceding frame by one or more cores in a multi-core processor, and a slice is processed in a succeeding frame provided at least one of said one or more cores in the multi-core processor is available for processing; and the processing of a slice from said one or more slices at same physical location in the preceding input frame has been completed, the processing of frames and slices in the preceding and succeeding frames being performed at least partially in parallel. A significant speedup in comparison to the sequential encoding approach is achieved while maintaining high visual quality for the output video. A corresponding system for parallel encoding of a video is also provided.

Claims

1. A method for encoding a video, comprising:

(a) encoding two successive input frames of the video, a preceding input frame and a succeeding input frame, comprising;

(a1) dividing the preceding input frame into slices, each slice characterizing a spatial region within the preceding input frame;

(a2) dividing the succeeding input frame into slices, each slice characterizing a spatial region within the succeeding input frame;

(a3) processing one or more slices in the preceding input frame by one or more cores in a multi-core processor or by one or more hardware processors;

(a4) processing a slice in the succeeding input frame provided;

(i) at least one of said one or more cores in the multi-core processor or said one or more hardware processors is available for processing; and

(ii) the processing of a slice from said one or more slices at same physical location in the preceding input frame has been completed;

wherein the steps (a3) and (a4) are performed at least partially in parallel.

2. The method of claim 1, wherein the preceding input frame and the succeeding input frame is one of the intra frame and predicted frame.

3. The method of claim 1, wherein the preceding input frame and the succeeding input frame are predicted frames.

4. The method of claim 1, wherein two or more steps (a1)-(a4) are performed at least partially in parallel.

5. The method of claim 1, further comprising performing the step (a4) immediately upon the condition (ii) is met.

6. The method of claim 5, further comprising performing motion estimation for the slice in the succeeding input frame using said slice from said one or more slices at same physical location in the preceding input frame.

7. The method of claim 1, further comprising:

selecting a motion estimation window containing said one or more slices; and

performing the step (a4) provided;

(iii) the processing of said one or more slices contained within the motion estimation window has been completed, the motion estimation window also containing said slice at the same physical location in the preceding frame.

8. The method of claim 7, further comprising performing a motion estimation for the slice in the succeeding input frame using said one or more slices contained within the motion estimation window.

9. The method of claim 7, wherein said one or more slices within the motion estimation comprise one of the following:

at least one of the slices on top and at bottom of said slice at the same physical location in the preceding input frame;

both a slice on top and a slice at bottom of said slice at the same physical location in the preceding input frame;

more than one slice above said slice at the same physical location in the preceding input frame;

more than one slice below said slice at the same physical location in the preceding input frame; and

all slices within the motion estimation window.

10. The method of claim 9, further comprising performing a motion estimation for the slice in the succeeding input frame using:

said slice at the same physical location in the preceding input frame; and

one of the following;

at least one of the slices on top and at bottom of said slice at the same physical location in the preceding input frame;

both a slice on top and a slice at bottom of said slice at the same physical location in the preceding input frame;

more than one slice above said slice at the same physical location in the preceding input frame;

more than one slice below said slice at the same physical location in the preceding input frame; and

all other slices within the motion estimation window.

11. The method of claim 1, further comprising:

for each input frame, generating a corresponding output frame comprising processed slices; and

writing the output frames in an output video.

12. The method of claim 1, wherein a sum of a number of slices in the preceding input frame and a number of slices in the succeeding input frame is equal to or greater than a number of said one or more cores or said one or more hardware processors.

13. The method of claim 1, wherein:

the step (a3) further comprises performing a pre-slice encoding of the preceding input frame before processing said one or more slices in the preceding input frame, comprising writing a frame type and a frame header into a respective output frame; and

the step (a4) further comprises performing a pre-slice encoding of the succeeding input frame before processing the slice in the succeeding input frame, comprising writing a frame type and a frame header into a respective output frame;

the pre-slice encoding of the succeeding input frame being performed after the pre-slice encoding of the preceding input frame is completed.

14. The method of claim 1, wherein:

the step (a3) further comprises performing a post-slice encoding of the preceding input frame after processing all slices in the preceding input frame, comprising applying a deblocking filter for smoothing sharp edges formed between macroblocks of the preceding input frame; and

the step (a4) further comprises performing a post-slice encoding of the succeeding input frame after processing all slices in the succeeding input frame, comprising applying a deblocking filter for smoothing sharp edges formed between macroblocks of the succeeding input frame;

the post-slice encoding of the succeeding input frame being performed after the post-slice encoding of the preceding input frame is completed.

15. The method of claim 12, wherein the sum of a number of slices in the preceding input frame and a number of slices in the succeeding input frame is about one and a half times the number of cores or said one or more hardware processors.

16. The method of claim 1, wherein:

the step (a3) further comprises performing quantization for the preceding input frame producing a quantization parameter for motion estimation for said one or more slices in the succeeding input frame and rate distortion optimization; and

the step (a4) further comprises performing quantization for the succeeding input frame by using the quantization parameter.

17. The method of claim 11, wherein the output video is in a H.264 format.

18. A non-transitory computer readable storage medium, having a computer readable program code instructions stored thereon, which, when executed by a processor, perform the following:

(a) encoding two successive input frames of the video, a preceding input frame and a succeeding input frame, comprising;

(a1) dividing the preceding input frame into slices, each slice characterizing a spatial region within the preceding input frame;

(a2) dividing the succeeding input frame into slices, each slice characterizing a spatial region within the succeeding input frame;

(a3) processing one or more slices in the preceding input frame by one or more cores in a multi-core processor or by one or more hardware processors;

(a4) processing a slice in the succeeding input frame provided;

(i) at least one of said one or more cores in the multi-core processor or said one or more hardware processors is available for processing; and

(ii) the processing of a slice from said one or more slices at same physical location in the preceding input frame has been completed;

wherein the computer readable instructions cause the processor to process said one or more slices in the preceding frame and said slice in the succeeding frame at least partially in parallel.

19. A computerized system for encoding a video comprising:

a processor, and a non-transitory computer readable storage medium having computer readable instructions stored thereon for execution by the processor, forming;

(a) an encoding unit encoding two successive input frames of the video, a preceding input frame and a succeeding input frame, the encoding unit comprising;

(a1) a slicer module dividing the preceding input frame into slices characterizing respective spatial regions within the preceding input frame, and the succeeding input frame into slices characterizing respective spatial regions within the succeeding input frame;

(a2) a slice processing module, processing one or more slices in the preceding input frame and one or more slices in the succeeding input frame, by one or more cores in a multi-core processor or by one or more hardware processors, provided;

(i) at least one of said one or more cores in the multi-core processor or said one or more hardware processors is available for processing; and

(ii) the processing of a slice from said one or more slices at same physical location in the preceding input frame has been completed;

wherein the slice processing module (a2) is configured to process said one or more slices in the preceding input frame and said one or more slices in the succeeding input frame at least partially in parallel.

20. The system of claim 19, wherein the preceding input frame and the succeeding input frame is one of the intra frame and predicted frame.

21. The system of claim 19, wherein the preceding input frame and the succeeding input frame are predicted frames.

22. The system of claim 19, wherein the slicer module and the slice processing module are configured to process two or more of the following at least partially in parallel:

the dividing the preceding input frame into slices;

the dividing the succeeding input frame into slices;

the processing one or more slices in the preceding input frame; and

the processing one or more slices in the succeeding input frame.

23. The system of claim 19, wherein the computer readable instructions cause the processor to perform the processing a slice in the succeeding input frame immediately upon the condition (ii) is met.

24. The system of claim 23, further comprising a motion estimation module performing motion estimation of the slice in the succeeding input frame using said slice from said one or more slices at same physical location in the preceding input frame.

25. The system of claim 19, wherein the computer readable instructions further cause the processor to:

select a motion estimation window containing said one or more slices; and

perform the processing a slice in the succeeding input frame provided;

(iii) the processing of said one or more slices contained within the motion estimation window has been completed, the motion estimation window also containing said slice at the same physical location in the preceding input frame.

26. The system of claim 25, wherein the motion estimation module is further configured to perform a motion estimation of the slice in the succeeding input frame using said one or more slices contained within the motion estimation window.

27. The system of claim 25, wherein said one or more slices within the motion estimation window comprise one of the following:

at least one of the slices on top and at bottom of said slice at the same physical location in the preceding input frame;

both a slice on top and a slice at bottom of said slice at the same physical location in the preceding input frame;

more than one slice above said slice at the same physical location in the preceding input frame;

more than one slice below said slice at the same physical location in the preceding input frame; and

all slices within the motion estimation window.

28. The system of claim 27, wherein the motion estimation module is further configured to perform a motion estimation of the slice in the succeeding input frame using:

said slice at the same physical location in the preceding input frame; and

one of the following;

at least one of the slices on top and at bottom of said slice at the same physical location in the preceding input frame;

both a slice on top and a slice at bottom of said slice at the same physical location in the preceding input frame;

more than one slice above said slice at the same physical location in the preceding input frame;

more than one slice below said slice at the same physical location in the preceding input frame; and

all other slices within the motion estimation window.

29. The system of claim 19, further comprising an output frame generator module, generating

for each input frame, a corresponding output frame comprising processed slices;

and

a writer module writing the output frames in an output video.

30. The system of claim 19, wherein a sum of a number of slices in the preceding input frame and a number of slices in the succeeding input frame is equal to or greater than a number of said one or more cores or said one or more hardware processors.

31. The system of claim 19, wherein the slice processing module further comprises a pre-slice encoding module configured to:

perform a pre-slice encoding of the preceding input frame before processing said one or more slices in the preceding input frame, comprising writing a frame type and a frame header into a respective output frame; and

perform a pre-slice encoding of the succeeding input frame before processing the slice in the succeeding input frame, comprising writing a frame type and a frame header into a respective output frame;

the pre-slice encoding of the succeeding input frame being performed after the pre-slice encoding of the preceding input frame is completed.

32. The system of claim 19, wherein the slice processing module further comprises a post-slice encoding module configured to:

perform a post-slice encoding of the preceding input frame after processing all slices in the preceding input frame, comprising applying a deblocking filter for smoothing sharp edges formed between macroblocks of the preceding input frame; and

perform a post-slice encoding of the succeeding input frame after processing all slices in the succeeding input frame, comprising applying a deblocking filter for smoothing sharp edges formed between macroblocks of the succeeding input frame;

the post-slice encoding of the succeeding input frame being performed after the post-slice encoding of the preceding input frame is completed.

33. The system of claim 30, wherein the sum of a number of slices in the preceding input frame and a number of slices in the succeeding input frame is about one and a half times the number of said cores or said one or more hardware processors.

34. The system of claim 19, wherein the slice processing module further comprises a quantization module configured to:

perform quantization for the preceding input frame to produce a quantization parameter; and

perform quantization for the succeeding input frame by using the quantization parameter.

35. The system of claim 29, wherein the output video is in a H.264 format.