Multithread processing of video frames

• US 8,223,845 B1
• Filed: 03/16/2005
• Issued: 07/17/2012
• Est. Priority Date: 03/16/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Duvivier, Christian L.
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): HALLENBECK-HUBER, JEREMIAH CHARLES
• Application Number:
  US11/083,630
• Time in Patent Office: 2,680 Days
• Field of Search: 375/240.24
• US Class Current: 375/240.24
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

Method for determining frame slice sizes of a frame for multithreaded decoding. The frame is encoded using at least two different slice types based on size where a large-type slice is at least two times larger than a small-type slice and/or the large-type slices comprise 70-90% of the frame. In some embodiments, the number of large-type slices is equal to the number of threads available for decoding and comprise the beginning slices of the frame to be decoded before the small-type slices. Methods for multithreaded deblocking of the frame under the H.264 codec is provided where first and second threads processes first and second sections of the frame in parallel. The first section comprises macroblocks on one side of a diagonal line and the second section comprises the remainder, the diagonal line extending from a first corner of a sub-frame to a second corner of the sub-frame.

Claims

1. A method for encoding a video picture, the method comprising:

determining first and second size categories by setting a first number of slices for the first size category and setting a second number of slices for the second size category, wherein the first number of slices is equal to a number of parallel task threads available to encode slices of the video picture and the second number of slices is based on a total number of slices for the video picture and the first number of slices;

dividing the video picture into a plurality of different sized slices that are categorized into the first and second size categories, wherein at least one slice belongs to each of the first and second categories, wherein each of the slice sizes is specified as a number of macroblocks in the slice; and

at an encoder, encoding the video picture by starting encoding of all slices in the first size category before starting encoding of any slice in the second size category.

2. The method of claim 1, wherein a ratio between the slice sizes of the first and second categories is larger than a minimum ratio.

3. The method of claim 1, wherein two or more slices of a same size category are different in size.

4. The method of claim 1, wherein all slices in the first size category are larger in size than all slices in the second size category.

5. The method of claim 4 further comprising determining a slice size of the first category and a slice size of the second category such that the percentage of the video picture that is to be encoded as slices in the first category is larger than a minimum percentage.

6. The method of claim 4, wherein the second number of slices is equal to the total number of slices for the video picture minus the first number of slices.

7. The method of claim 4, wherein the first size category comprises slices that are configured to be decoded before the slices in the second size category.

8. A non-transitory computer readable medium storing a computer program for execution by at least one processing unit, the computer program for encoding a video frame, the computer program comprising sets of instructions for:

identifying a plurality of size categories that specify different video frame slice sizes, wherein at least one size category specifies two or more different slice sizes;

setting a number of slices in a first size category equal to a number of parallel task threads available to encode slices of the video frame;

setting a number of slices in a second size category based on a total number of slices for the video frame and the number of slices in the first size category; and

encoding the video frame by (i) using the identified different size categories to divide the video frame into a plurality of different sized slices and (ii) starting encoding of all slices in one size category before starting encoding of any slice in any other size category, wherein at least one slice belongs to each of the first and second categories, wherein each of the slice sizes is specified as a number of macroblocks in the slice.

9. The computer readable medium of claim 8, wherein all slices in the first size category are larger in size than all slices in the second size category.

10. The computer readable medium of claim 8, wherein a ratio between the slice sizes of the first and second size categories is larger than a minimum ratio.

11. The computer readable medium of claim 9, wherein the set of instructions for identifying the plurality of size categories comprises a set of instructions for determining a slice size of the first size category and a slice size of the second size category such that a percentage of the video frame that is to be encoded as slices in the first size category is larger than a minimum percentage.

12. The computer readable medium of claim 9, wherein the computer program further comprises sets of instructions for:

setting the number of slices in the second size category to equal the total number of slices for the video frame minus the number of slices in the first size category.

13. A method for decoding a video frame, the method comprising:

determining first and second size categories by setting a number of slices in the first size category equal to a number of available parallel task threads to decode slices of the video frame and setting a number of slices in the second size category based on a total number of slices in the video frame and the number of slices in the first size category;

receiving an encoded video frame having a plurality of slices; and

at a decoder, decoding the encoded frame by using the available parallel task threads to start decoding of all slices in the first size category before starting decoding of any slice in the second size category, wherein at least one slice belongs to each of the first and second categories, wherein each of the slice sizes is specified as a number of macroblocks in the slice.

14. The method of claim 13, wherein two or more slices of a same size category are different in size.

15. The method of claim 13, wherein all slices in the first size category are larger in size than all slices in the second size category.

16. The method of claim 15, wherein a ratio between a slice size of the first size category and a slice size of the second size category is larger than a minimum ratio.

17. The method of claim 15, wherein the number of slices in the second size category is equal to the total number of slices for the video frame minus the number of slices in the first size category.

18. A non-transitory computer readable medium storing a computer program for execution by at least one processing unit, the computer program for decoding a video frame, the computer program comprising sets of instructions for:

identifying, based on a set of available computing resources, a plurality of size categories that specify a plurality of different video frame slice sizes;

setting a number of slices in a first size category equal to a number of parallel task threads available to decode the slices of the video frame;

setting a number of slices in a second size category based on a total number of slices for the frame and the number of slices in the first size category;

receiving an encoded video frame comprising a plurality of slices; and

decoding the encoded video frame by (i) associating each of the plurality of slices in the received video frame with one of the identified size categories and (ii) starting decoding of all slices associated with one size category before starting decoding of any slice associated with any other size category, wherein at least one size category is associated with slices of two or more different sizes, wherein at least one slice belongs to each of the first and second categories, wherein each of the slice sizes is specified as a number of macroblocks in the slice.

19. The computer readable medium of 18, wherein all slices in the first size category are larger in size than all slices in the second size category.

20. The computer readable medium of claim 19, wherein at least one slice associated with the second size category references a slice associated with the first size category for decoding.

21. The computer readable medium of claim 19, wherein the computer program further comprises sets of instructions for:

setting the number of slices in the second size category to equal the total number of slices for the frame minus the number of slices in the first size category.