Video processing system and method with dynamic tag architecture

• US 20060153473A1
• Filed: 07/15/2005
• Published: 07/13/2006
• Est. Priority Date: 01/13/2005
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Ruggiero, Carl J.
• Granted Patent: US 7,853,044 B2
• Time in Patent Office: Days
• US Class Current: 382/276
• CPC Class Codes: H04N 21/42692 for reading from or writing on a volatile storage medium, e.g. Random Access Memory [RAM]
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 5/46 for receiving on more than one standard at will deflecting circuits of multi-standard receivers H04N3/27
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229
• CPC Class Codes: H04N 7/0135 involving interpolation processes interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59
• CPC Class Codes: H04N 7/0145 the interpolation being class adaptive, i.e. it uses the information of class which is determined for a pixel based upon certain characteristics of the neighbouring pixels
• CPC Class Codes: H04N 9/64 Circuits for processing colour signals H04N9/77 takes precedence; camera processing pipelines for processing colour signals H04N23/84

Abstract

An image processing system including a plurality of image processors. Each image processor is configured to carry out a different image processing operation. The system also includes a motion estimator. For image processor, the motion estimator is configured to obtain motion information for a pixel or pixels being applied as a processing input to the image processor.

Claims

1. An image processing system, comprising:

a plurality of image processing modules, each being configured to carry out a different image processing operation; and

a motion estimation module operably coupled with and shared by the image processing modules, where, for each image processing module, the motion estimation module is configured to obtain motion information for a pixel or pixels being applied as a processing input to the image processing module.

2. The system of claim 1, where one or more of the image processing operations is dynamically controlled based on motion information obtained from the motion estimation module.

3. The system of claim 1, further comprising controller and classifier logic operatively coupled with the plurality of image processing modules and configured to control operation of the image processing modules, the controller and classifier logic being further configured to repeatedly update tag data associated with pixels of video data being processed by the image processing system, where said repeatedly updated tag data is used to dynamically control operation of the plurality of image processing modules.

4. The system of claim 3, where the tag data includes classification information pertaining to characteristics of one or more pixels contained in the pixels of video data.

5. The system of claim 3, where the tag data includes processing information pertaining to processing operations applied to one or more pixels contained in the pixels of video data.

6. The system of claim 3, where the tag data includes source identification information associated with one or more pixels contained in the pixels of video data.

7. The system of claim 1, where the plurality of image processing modules are configured to sequentially process pixels of video data such that the pixels of video data are sequentially and successively processed by each of the plurality of image processing modules.

8. The system of claim 7, where the plurality of image processing modules define a sequential processing pipeline capable of processing video data in a FIFO manner, such that during operation, multiple frames of video data are being processed within the sequential processing pipeline.

9. The system of claim 8, further comprising controller and classifier logic configured to dynamically and unequally allocate processing time for the image processing modules.

10. The system of claim 9, where the controller and classifier logic is configured to allocate processing time from one of the multiple frames of video data to another of the multiple frames of video data.

11. The system of claim 9, where the controller and classifier logic is configured to allocate processing time from one portion of one of the multiple video frames to another portion of said video frame.

12. The system of claim 9, where the controller and classifier logic is configured to allocate processing time from one of the plurality of image processing modules to another of the plurality of image processing modules.

13. An image processing system, comprising:

a deinterlacing module configured to receive interlaced video input, and to output progressive scan full vertical resolution video;

an image interpolation module configured to effect at least one of supersampling and subsampling; and

a motion estimation module operatively coupled with and shared by the deinterlacing module and the image interpolation module, where the motion estimation module is configured to obtain motion information for a pixel or pixels being applied as inputs to the deinterlacing module and the image interpolation module, and where deinterlacing and interpolation operations are controlled in part based on such motion information.

14. The system of claim 13, further comprising controller and classifier logic operatively coupled with and configured to control operation of the deinterlacing module and image interpolation module, where for a given group of pixels, the controller and classifier logic is configured to update tag data for the given group of pixels between operation of the deinterlacing module and the image interpolation module on the given group of pixels, so as to yield updated tag data, where the tag data and the updated tag data are used to dynamically control deinterlacing and interpolation operations.

15. An integrated circuit, comprising image processing logic configured to execute first and second image processing operations on one or more pixels of video data, where the image processing logic includes at least one shared logic module used to execute an image processing routine common to the first and second image processing operations.

16. An image processing system, comprising:

a plurality of image processing modules, each being configured to carry out a different image processing operation; and

a motion estimation module operably coupled with and shared by the image processing modules, where, for each image processing module, the motion estimation module is configured to obtain motion information for a pixel or pixels being applied as a processing input to the image processing module,

where the image processing system is configured to dynamically control each of the plurality of image processing modules based on the motion information.

17. An integrated circuit, comprising:

image processing logic configured to perform multiple different image processing operations on pixels of video data, where at least some of the multiple image processing operations are dynamically controllable based on motion information associated with pixels to be processed by the at least some of the multiple image processing operations; and

motion estimation logic operatively coupled with and shared by the at least some of the multiple image processing operations, the motion estimation logic being configured to obtain and provide motion information for use in effecting dynamic control of the at least some of the multiple different image processing operations.

18. The integrated circuit of claim 17, further comprising controller and classifier logic configured to dynamically control the multiple different image processing operations based on dynamically changing tag data associated with the pixels of video data.

19. The integrated circuit of claim 18, where the tag data includes classification information pertaining to characteristics of one or more pixels contained in the pixels of video data.

20. The integrated circuit of claim 18, where the tag data includes processing information pertaining to processing operations applied to one or more pixels contained in the pixels of video data.

21. The integrated circuit of claim 18, where the tag data includes source identification information associated with one or more pixels contained in the pixels of video data.

22. The integrated circuit of claim 18, where the image processing logic and controller and classifier logic are configured to execute one of the multiple different image processing operations by:

dynamically selecting, for each pixel or group of pixels to be processed by said one of the multiple different image processing operations, a particular implementation of said image processing operation; and

applying the particular implementation to the pixel or group of pixels,

where the particular implementation is selected from a plurality of loaded implementations of said image processing operation, and where the plurality of loaded implementations is varied from time to time during operation of the integrated circuit.

23. A method of processing video data, comprising:

receiving pixels of video data;

using a motion estimation module to perform a temporal analysis on the pixels of video data to obtain motion information;

performing a first image processing operation on the pixels of video data, the first image processing operation being dynamically controlled based of the motion information;

after performance of the first image processing operation, using the motion estimation model to perform an additional temporal analysis on the pixels of video data, the additional temporal analysis being performed to update motion information for use in dynamic control of a second image processing operation which is different from the first image processing operation; and

performing the second image processing operation on the pixels of video data;

the second image processing operation being dynamically controlled based on updated motion information obtained from the motion estimation module.

24. An integrated circuit, comprising:

image processing logic configured to perform multiple different image processing operations on pixels of video data, where at least some of the multiple image processing operations are dynamically controllable based on motion information associated with pixels to be processed by the at least some of the multiple image processing operations; and

motion estimation logic operatively coupled with and shared by the at least some of the multiple image processing operations, the motion estimation logic being configured to obtain and provide motion information for use in effecting dynamic control of the at least some of the multiple different image processing operations, where the motion estimation logic is operative to obtain motion information by selecting and applying a selected motion estimation routine to the pixels of video data, the selected motion estimation routine being selected from a plurality of motion estimation routines loaded into a memory location.

25. The integrated circuit of claim 24, where the integrated circuit includes controller logic configured to monitor operation of the motion estimation logic and calibrate the motion estimation logic from time to time based on such monitoring, where said calibration includes unloading one or more of the plurality of motion estimation routines and replacing such unloaded routine or routines with a different motion estimation routine or routines.