Accelerated video encoding using a graphics processing unit

• US 7,813,570 B2
• Filed: 10/22/2004
• Issued: 10/12/2010
• Est. Priority Date: 09/13/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Shipeng, Gao, Guangping, Shen, Guobin
• Primary Examiner(s): Mariam; Daniel G
• Assistant Examiner(s): Woldemariam; Aklilu k
• Application Number:
  US10/971,545
• Publication Number:
  US 20060056513A1
• Time in Patent Office: 2,181 Days
• Field of Search: 382/236, 382/303, 382/304, 382/302, 356/336, 375/240.12, 375/240.16, 375/240.24, 376/310, 428/64.1, 428/32.32
• US Class Current: 382/236
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/557 Motion estimation characterised by stopping computation or iteration based on certain criteria, e.g. error magnitude being too large or early exit
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]

Abstract

A video encoding system uses both a central processing unit (CPU) and a graphics processing unit (GPU) to perform video encoding. The system implements a technique that enables the GPU to perform motion estimation for video encoding. The technique allows the GPU to perform a motion estimation process in parallel with the video encoding process performed by the CPU. The performance of video encoding using such a system is greatly accelerated as compared to encoding using just the CPU. Also, data related to motion estimation is arranged and provided to the GPU in a way that utilizes the capabilities of the GPU. Data about video frames may be collocated to enable multiple channels of the GPU to process tasks in parallel. The depth buffer of the GPU may be used to consolidate repeated calculations and searching tasks during the motion estimation process.

Claims

1. A method for processing a digital video signal with a graphics processing unit (GPU) having multiple channels and a central processing unit (CPU), the method comprising:

identifying a frame associated with the video signal by the CPU;

determining motion estimation data by the GPU, the determining comprising;

collocating the frame to create copies of the frame such that each of the copies of the frame is available for processing in parallel by at least one of the multiple channels of the GPU, and to map each of a plurality of pixels selected by the GPU to each of a plurality of channels of a texel; and

processing each of the copies of the frame in parallel, using a different channel of the multiple channels of the GPU, the processing including offsetting the copies of the frame by a predetermined value;

outputting the determined motion estimation data to the CPU, and

encoding the video signal by the CPU,

wherein the determining motion estimation data by the GPU is performed in parallel to the encoding the video signal performed by the CPU.

2. The method as recited in claim 1, wherein the collocating the frame includes creating four padded copies of the frame for parallel processing by four channels of the graphics processing unit.

3. The method as recited in claim 1, further comprising padding each of the copies of the frame with a horizontal band of pixels and a vertical band of pixels.

4. The method as recited in claim 1, further comprising dividing each of the copies of the frames into macroblocks.

5. The method as recited in claim 4, wherein each of the macroblocks is at least one of 16 pixels by 16 pixels, 8 pixels by 8 pixels, or 4 pixels by 4 pixels.

6. The method as recited in claim 4, further comprising shifting pixels of the macroblocks such that the macroblocks for a current one of the copies of the frame is displaced by a pixel offset relative to a previous one of the copies of the frame.

7. The method as recited in claim 6, wherein the pixel offset includes a range of 1 to 16 pixels.

8. The method as recited in claim 4, further comprising mapping pixels in the macroblocks of each of the copies of the frame to channels of the graphics processing unit.

9. The method as recited in claim 4, further comprising mapping pixels in the macroblocks of each of the copies of the frame to texels that can be processed by a shader of the graphics processing unit.

10. The method as recited in claim 9, further comprising calculating motion estimation data for each macroblock using the texels.

11. The method as recited in claim 10, wherein the motion estimation data includes at least one of a motion vector, a sum of absolute difference (SAD) value, a mean square error (MSE) value, or a R-D based metric value.

12. The method as recited in claim 9, wherein a group of 32 texels are processed by the shader at a time.

13. The method as recited in claim 9, further comprising dividing each macroblock into texel blocks.

14. The method as recited in claim 13, wherein each texel block includes at least one of a 4.times.2 or a 2.times.4 pixel block.

15. The method as recited in claim 13, further comprising calculating motion estimation data for each texel block, wherein the motion estimation data includes at least one of a sum of absolute difference (SAD) value, a mean square error (MSE) value, or a R-D based metric value.

16. The method as recited in claim 15, further comprising summing up the motion estimation data for the texel blocks associated with each macroblock.

17. The method as recited in claim 16, further comprising determining the motion estimation data associated with the digitized frame from the motion estimation data associated with the macroblocks in the digitized frame.

18. An apparatus for processing video signal comprising:

a central processing unit configured to receive motion estimation data associated with the video signal and to encode the video signal into digital video data using the motion estimation data; and

a graphics processing unit including multiple channels, the graphics processing unit configured to identify a particular frame associated with the video signal, to collocate the particular frame into collocated frames, such that each of the collocated frames is a copy of the particular frame, available for processing in parallel by each channel of the graphics processing unit, and to map the each of a plurality of pixels selected by the graphics processing unit to each of a plurality of channels of a texel so that each channel of the graphics processing unit can parallel process each of the collocated frames to obtain the motion estimation data, the parallel processing including offsetting the collocated frames by a predetermined value,

wherein the obtaining motion estimation data by the graphics processing unit is performed in parallel to the encoding the video signal performed by the central processing unit.

19. The apparatus as recited in claim 18, wherein the graphics processing unit is further configured to divide each collocated frame into macroblocks.

20. The apparatus as recited in claim 19, wherein the graphics processing unit is further configured to shift pixels of the macroblocks such that the macroblocks for a current collocated frame are displaced by a pixel offset relative to the macroblocks of a previous one of the collocated frames.

21. The apparatus as recited in claim 19, wherein the graphics processing unit is further configured to divide corresponding pixels from each of the collocated frames into the texel blocks so that the pixels from multiple collocated frames can be processed in parallel by the multiple channels of the graphics processing unit.

22. The apparatus as recited in claim 21, wherein the graphics processing unit includes a shader and wherein the shader of the graphics processing unit is configured to process the texel blocks.

23. The apparatus as recited in claim 21, wherein the graphics processing unit is further configured to determine motion estimation data for each of the texel blocks and wherein the motion estimation data includes at least one of a sum of absolute difference (SAD) value, a mean square error (MSE) value, or a R-D based metric value.

24. The apparatus as recited in claim 18, wherein the graphics processing unit is further configured to determine motion vectors for the particular frame from processing the texel blocks and to provide the motion vectors to the central processing unit.

25. The apparatus as recited in claim 24, wherein the central processing unit is further configured to encode the particular frame into the digital video data using the motion vectors.

26. A System for encoding video frames into digital video data, the system, comprising:

a processor;

One or more memories coupled to the processor, the one or more memories having stored instructions that, when executed at the system, configure the system to implement;

a first data field including data that represents collocated digitized frames which are copies of a particular frame;

a second data field indexed to the first data field, the second data field including data representing multiple sets of macroblocks, each set of macroblocks corresponding to each of the collocated digitized frames of the first data field; and

a third data field indexed to the second data field, the third data field including data that represents groups of pixels in the macroblocks, each group of pixels including pixels from each set of corresponding macroblocks, each group of pixels being arranged as texel blocks for utilization by channels of a graphics processing unit to process the texel blocks in parallel,

the data of the third data field being arranged to be processed by the graphics processing unit to determine motion estimation data, the arrangement including a predetermined pixel offset of at least one of the groups of pixels and a mapping of a selected group of pixels to channels of a texel,

the data of the third data field being further arranged to be processed by the graphics processing unit to determine motion estimation data in parallel with the processing device encoding a video signal based on the frames.

27. The System as recited in claim 26, wherein at least one of the groups of pixels are arranged as texel blocks that can be inputted into the channels of the graphics processing unit.

28. The System as recited in claim 27, wherein the groups of pixels are arranged as four texel blocks and wherein each texel blocks are inputted into a particular channel of the graphics processing unit.

29. An apparatus for encoding video frames into digital video data comprising:

means for collocating a particular video frame into collocated frames to create copies of the frame such that each of the copies of the frame is available for processing in parallel by at least one of multiple channels of a graphics processing unit (GPU), and to map each of a plurality of pixels selected by the GPU to each of a plurality of channels of a texel;

means for dividing each of the collocated frames into texel blocks; and

means for providing the texel blocks to each channel of the GPU so that the collocated frames can be processed in parallel to obtain motion estimation data associated with the video frame, the processing including offsetting the copies of the frame by a predetermined value,

wherein the means for providing is configured to obtain the motion estimation data by the GPU in parallel with encoding the video frames by a central processing unit.

30. The apparatus of claim 29, further comprising:

means for determining motion estimation data for each of the texel blocks; and

means for determining overall estimation data for each of the macroblocks.

31. The apparatus of claim 29, further comprising:

means for determining motion vectors associated with the particular frame based, at least in part, on the motion estimation data for each of the macroblocks; and

means for encoding the particular frame into the digital video data using the motion vectors.