Video image processing with parallel processing

• US 9,292,904 B2
• Filed: 07/15/2005
• Issued: 03/22/2016
• Est. Priority Date: 01/16/2004
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Ruggiero, Carl J.
• Primary Examiner(s): TUCKER, WESLEY J
• Application Number:
  US11/183,160
• Publication Number:
  US 20060018562A1
• Time in Patent Office: 3,903 Days
• Field of Search: 382302-304, 382/307
• US Class Current: 1/1
• CPC Class Codes: G06F 9/3879 for non-native instruction execution, e.g. executing a command; for Java instruction set
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: H04N 5/205 for correcting amplitude versus frequency characteristic
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229
• CPC Class Codes: H04N 7/0135 involving interpolation processes interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59
• CPC Class Codes: H04N 7/0137 dependent on presence/absence of motion, e.g. of motion zones H04N7/014 takes precedence; movement detection in television signals H04N5/144

Abstract

This document discusses systems and methods that track overall time for processing operations such that the processing time can be shared among the resources efficiently. Processing time can be shifted to image processing to provide the most benefit to image quality. Moreover, access time from one process is banked to be used by a subsequent process or on a subsequent group of pixels. This document discusses systems and methods that provide additional processing power on an as needed basis. For example, a processing stage and its controller are outside the normal pixel processing flow path. When it is determined that additional processing is required, the processing stage and its controller are activated to perform the additional processing. This document discusses systems and methods that provide parallel processing in a processing stage such that the data can flow internal to the controller linked to the processing stage and globally.

Claims

1. A digital video signal processing system comprising:

memory;

a main processor;

a processing pipeline coupled to said main processor and said memory, wherein said processing pipeline comprises a plurality of processing segments coupled to said a memory, wherein a respective processing segment comprises;

a stage controller configured to issue an instruction for performing a processing task on a digital video signal; and

a processing unit coupled to said stage controller and configured to perform said processing task on said digital video signal responsive to said instruction,

wherein said stage controller is configured to perform one or more other operations in parallel with said processing unit performing said processing task,

wherein said one or more other operations are completed at said stage controller before said processing task is completed at said processing unit, and

wherein said processing unit is configured to send an indication to said stage controller, wherein said indication is indicative of a completion of said processing task.

2. The system of claim 1, wherein said plurality of processing segments comprise processing units respectively configured as a deinterlacer, an image interpolator, a color processor, an image enhancer/booster, and wherein each of said processing units is coupled to a separate stage controller.

3. The system of claim 1, wherein the stage controller is configured to stall after performing said one or more other operations to wait for said processing unit to complete said processing task.

4. The system of claim 1, wherein said stage controller comprises a bridge coupled between said stage controller and the memory.

5. The system of claim 3, wherein said stage controller comprises internal-memory configured to store a process time of said processing task.

6. The system of claim 1, wherein said respective processing segment is configured to generate processed digital video signal and send said processed digital video signal to a next processing segment.

7. The system of claim 6, wherein said processed digital video signal comprises pixel data and tag data.

19. The system of claim 1, wherein said plurality of processing segments comprises an on-demand segment configured to provide processing power on demand from any other processing segment of said plurality of processing segments.

20. The system of claim 1, wherein said one or more other operations are independent of said processing task.

21. The system of claim 1, wherein said respective processing segment is configured to send said processed digital video signal to said memory for access by a next processing segment.

8. A method of processing video images, said method comprising:

receiving image data at a first segment of a processing pipeline, wherein said processing pipeline comprises a plurality of segments, and wherein said first segment comprises a stage controller and a processor;

sending an instruction from said stage controller to said processor, wherein said instruction is operable to initiate a processing task to be performed by said processor;

subsequent to said sending, performing said one or more other operations at said stage controller in parallel with said performing said processing task, wherein said one or more other operations are completed at said stage controller before said processing task is completed at said processor; and

sending a signal from said processor to said stage controller, said signal indicative of a completion of said processing task.

9. The method of claim 8 further comprising accessing a duration for said stage controller to perform one or more other operations, wherein said duration is stored in a storage unit internal to said first segment.

10. The method of claim 8 further comprising causing said stage controller to stall subsequent to a completion of said one or more other operations and before receiving said signal.

11. The method of claim 10 further comprising, upon receiving said signal, sending said output image data from said first segment to a second segment of said plurality of processing pipeline for further processing and/or to a memory coupled to said plurality of processing pipeline.

12. The method of claim 8, wherein said one or more operations are selected from a group consisting of register updates, address updates, data movement between registers and memory, loading new coordinates for a motion search, and updating tag data.

13. The method of claim 9, wherein said one or more operations are independent of said processing task.

14. The method of claim 8 further comprising:

accessing first tag data from memory that is coupled to said plurality of segments;

generating second tag data at said first segment; and

outputting said tag data to said second segment.

15. The method of claim 14, wherein said tag data is operable to affect a processing time of another segment in said plurality of segments.

16. The method of claim 8, wherein said processing pipeline comprises a utility segment, and further comprising:

aborting said processing task at said processor if said processor is unable to complete said processing task;

sending processing instructions to said utility segment; and

activating said utility segment to perform said processing task.

17. The method of claim 16, wherein said utility segment is operable to receive processing tasks assigned from any of said plurality of segments.

18. The method of claim 8, wherein said processing task is selected from a group consisting of image enhancement, interlace-to-progressive digital signal reconstruction, determining film mode frame rate and sequence mapping of interlaced fields onto frames, image scaling, refining scaled pixel data, and color processing.