Bandwidth-efficient processing of video images

• US 7,139,002 B2
• Filed: 10/27/2003
• Issued: 11/21/2006
• Est. Priority Date: 08/01/2003
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Estrop, Stephen J.
• Primary Examiner(s): Tung, Kee M.
• Assistant Examiner(s): RICHER, JONI
• Application Number:
  US10/694,144
• Publication Number:
  US 20050024363A1
• Time in Patent Office: 1,121 Days
• Field of Search: 345/542, 345/522, 345501-506, 345/582, 345/540, 345/581, 345/591
• US Class Current: 345/501
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2310/0229 De-interlacing
• CPC Class Codes: G09G 2340/0435 Change or adaptation of the frame rate of the video stream
• CPC Class Codes: G09G 2360/125 using unified memory architecture [UMA]
• CPC Class Codes: G09G 5/363 Graphics controllers
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229
• CPC Class Codes: H04N 7/0122 the input and the output signals having different aspect ratios

Abstract

A technique is described for performing multiple video processing tasks in a single operation, as opposed to serially. For instance, a technique is described for de-interlacing a principal video stream at the same time that at least one video sub-stream is combined with the principal video stream. Performing these tasks in a single call to a graphics processing unit, as opposed to staggered serial calls, reduces the bandwidth requirements of the processing operations. This, in turn, can enable a computing device to perform these multiple operations at full frame rate. In one implementation, different texturing units are respectively assigned to the principal video stream and the video sub-stream. The graphics processing unit interacts with these texturing units and associated memory locations substantially in parallel, thus providing the above-described bandwidth savings.

Claims

1. A method for processing video data in an apparatus including a computer processing module and a graphics processing module, comprising:

receiving a principal video stream from a source via plural inputs to the graphics processing module, the plural inputs associated with respective components of the principal video stream;

receiving a video sub-stream containing supplemental information associated with the principal video stream via another input to the graphical processing module;

in a single stage operation, performing, by the graphics processing module, an operation on the principal video stream and combining the principal video stream with the video sub-stream to produce processed data, wherein the single stage operation requires only a single read transaction to perform the single stage operation; and

outputting the processed data,

wherein each of the plural inputs for receiving the principal video stream and the other input for receiving the video sub-stream refer to separate inputs, and

wherein the graphics processing module executes video processing tasks using a 3D graphics pipeline.

2. The method according to claim 1, wherein the performing of the operation comprises de-interlacing the principal video stream.

3. The method according to claim 1, wherein the performing of the operation comprises resizing the principal video stream.

4. The method according to claim 1, wherein the performing and the is combining are performed in a YUV color space.

5. The method according to claim 1, further including a step of forwarding instructions to the graphics processing module, the instructions informing the graphics processing module how to execute the performing and the combining.

6. The method according to claim 5, wherein the instructions identify a location at which to receive the principal video stream, a location at which to receive the video sub-stream, and a location at which to provide the processed data.

7. The method according to claim 6, wherein the instructions identify a rectangle of data from which to receive the principal video stream within a video stream surface, a rectangle of data from which to receive the video sub-stream within a video sub-stream surface, and a rectangle at which to output the processed data within a destination surface.

8. The method according to claim 1, wherein the video sub-stream includes at least one of:

close captioned information;

DVD sub-picture information;

or PAL teletext information.

9. The method according to claim 1, wherein the performing and the combining are performed on an apparatus that uses a Uniform Memory Architecture (UMA) design.

10. The method of claim 1, wherein the single stage operation includes only one read transaction.

11. The method of claim 10, wherein the single stage operation includes only one write transaction.

12. The method of claim 1, wherein the graphics processing module includes multiple texturing units associated with the plural inputs and the other input.

13. An apparatus for processing video data, comprising:

a computer processing module for controlling the apparatus;

a renderer module;

a graphics processing module; and

an interface module that couples the renderer module to the data processing module,

wherein the renderer module includes logic configured to generate and provide instructions to the graphics processing module to execute at least the following functions in a single stage;

a) performing an operation on a principal video stream received via plural inputs to the graphics processing module, the plural inputs associated with respective components of the principal video stream; and

b) combining the received principal video stream with a video sub-stream, wherein the video sub-stream is received via another input to the graphics processing module,

wherein the single stage operation requires only a single read transaction to perform the single stage operation,

wherein each of the plural inputs for receiving the principal video stream and the other input for receiving the video sub-stream refer to separate inputs, and

wherein the graphics processing module is configured to execute video processing tasks using a 3D graphics pipeline.

14. An apparatus according to claim 13, wherein the performing of the operation comprises de-interlacing the principal video stream.

15. An apparatus according to claim 13, wherein the performing of the operation comprises resizing the principal video stream.

16. The apparatus according to claim 13, wherein the performing and the combining are performed in a YUV color space.

17. The apparatus according to claim 13, wherein the instructions provided by the renderer module inform the graphics processing module how to execute the performing and the combining.

18. The apparatus according to claim 17, wherein the instructions identify a location at which to receive the principal video stream, a location at which to receive the video sub-stream, and a location at which to provide the processed data.

19. The apparatus according to claim 18, wherein the instructions identify a rectangle of data from which to receive the principal video stream within a video stream surface, a rectangle of data from which to receive the video sub-stream within a video sub-stream surface, and a rectangle at which to output the processed data within a destination surface.

20. The apparatus according to claim 13, wherein the video sub-stream includes at least one of:

close captioned information;

DVD sub-picture information;

or PAL teletext information.

21. The apparatus according to claim 13, wherein the apparatus is configured to operate using a Uniform Memory Architecture (UMA) design.

22. The apparatus according to claim 13, wherein the graphics processing module includes multiple texturing units associated with the plural inputs and the other input.

23. The apparatus according to claim 22, wherein the graphics processing module is configured to execute the performing and the combining in a single stage by reading video data associated with the multiple texturing units at the same time.

24. An apparatus for processing video data, comprising:

a memory;

a computer processing module for controlling the apparatus, the computer processing module being coupled to the memory;

a renderer module;

a graphics processing module coupled to same memory as the computer processing module; and

an interface module that couples the renderer module to the graphics processing module,

wherein the renderer module includes logic configured to generate and provide instructions to the graphics processing module to execute at least the following functions in a single stage;

a) performing an operation on a principal video stream received via plural inputs to the graphics processing module, the plural inputs associated with respective components of the principal video stream; and

b) combining the received principal video stream with a video sub-stream, wherein the video sub-stream is received via another input to the graphics processing module,

wherein the graphics processing module includes logic configured to receive the instructions provided by the renderer module, and in response thereto, execute the performing and the combining,

wherein the single stage operation requires only a single read transaction to perform the single stage operation,

wherein each of the plural inputs for receiving the principal video stream and the other input for receiving the video sub-stream refer to separate inputs, and

wherein the graphics processing module is configured to execute video processing tasks using a 3D graphics pipeline.

25. The apparatus of claim 24, wherein the graphics processing module includes multiple texturing units associated with the plural inputs and the other input.

26. An apparatus for processing video data, comprising:

computer processing means for controlling the apparatus;

graphical processing means for performing graphical processing tasks, comprising;

means for receiving a principal video stream from a source via plural inputs to the graphical processing means, the plural inputs associated with respective components of the principal video stream;

means for receiving a video sub-stream containing supplemental information associated with the principal video stream via another input to the graphical processing means;

means, in a single stage operation, for performing an operation on the principal video stream and combining the principal video stream with the video sub-stream to produce processed data, wherein the single stage operation requires only a single read transaction to perform the single stage operation; and

means for outputting the processed data,

wherein each of the plural inputs for receiving the principal video stream and the other input for receiving the video sub-stream refer to separate inputs, and

wherein the graphics processing means is configured to execute video processing tasks using a 3D graphics pipeline.

27. The apparatus of claim 26, wherein the graphics processing means includes multiple texturing units associated with the plural inputs and the other input.

28. A computer readable media having machine readable instructions stored thereon for use by an apparatus including a computer processing module and a graphics processing module, the instructions comprising:

logic associated with the graphics processing module configured to receive a principal video stream from a source via plural inputs of the graphical processing module, the plural inputs associated with respective components to the principal video stream;

logic associated with the graphics processing module configured to receive a video sub-stream containing supplemental information associated with the principal video stream via another input to the graphical processing module;

logic associated with the graphics processing module configured to, in a single stage operation, perform an operation on the principal video stream and combine the principal video stream with the video sub-stream to produce processed data, wherein the single stage operation requires only a single read transaction to perform the single stage operation; and

logic configured to output the processed data,

wherein each of the plural inputs for receiving the principal video stream and the other input for receiving the video sub-stream refer to separate inputs, and

wherein the graphics processing module is configured to execute video processing tasks using a 3D graphics pipeline.

29. The computer readable media of claim 28, wherein the graphics processing module includes multiple texturing units associated with the plural inputs and the other input.

30. A method for processing video data using a graphics processing module, comprising:

receiving a principal video stream from a source via plural inputs to the graphics processing module, the plural inputs associated with respective components of the principal video stream;

receiving a video sub-stream containing supplemental information associated with the principal video stream via another input to the graphics processing module;

in a single stage operation, performing an operation on the principal video stream and combining the principal video stream with the video sub-stream to produce processed data, wherein the single stage operation involves reading first input data associated with the received principal video stream in parallel with second input data associated with the received video sub-stream data; and

outputting the processed data,

wherein each of the plural inputs for receiving the principal video stream and the other input for receiving the video sub-stream refer to separate inputs, and

wherein the graphics processing module executes video processing tasks using a 3D graphics pipeline.

31. The method of claim 30, wherein the graphics processing module includes multiple texturing units associated with the plural inputs and the other input.