Descriptor-based stream processor for image processing and method associated therewith

• US 9,241,142 B2
• Filed: 01/24/2013
• Issued: 01/19/2016
• Est. Priority Date: 01/24/2013
• Status: Active Grant

• Current Assignee: Analog Devices International Unlimited Company (Analog Devices, Inc.)
• Original Assignee: Analog Devices Global Unlimited Company (Analog Devices, Inc.)
• Inventors: Wang, Yong, Chen, Jianrong, Zhao, Yimiao, Huang, Youcheng, Chi, Xiaoming
• Primary Examiner(s): DAGNEW, MEKONNEN D
• Application Number:
  US13/749,675
• Publication Number:
  US 20140204232A1
• Time in Patent Office: 1,090 Days
• Field of Search: 348/241, 348/222.1, 382/261, 382/254
• US Class Current: 1/1
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

The present disclosure provides a stream processor, an associated stream controller and compiler, and associated methods for data processing, such as image processing. In some embodiments, a method includes defining a kernel pattern associated with an image frame, and processing the image frame using the defined kernel pattern. The method can further include generating a kernel switch lookup table based on the defined kernel pattern. In various implementations, the stream controller is operable to direct execution of kernels on the image frame according to the defined kernel pattern and the kernel switch lookup table.

Claims

1. A method for processing an image using a processor, wherein the processor can dynamically switch among multiple kernels, the method comprising:

defining a kernel pattern associated with an image frame by

grouping image pixels into a plurality of kernel blocks, wherein each kernel block has a respective kernel for performing at lest one operation on image pixels in the kernel block, and

determining a kernel sequence for the respective kernels; and

processing the image frame using the defined kernel pattern.

2. The method of claim 1, wherein defining the kernel pattern associated with the image frame includes generating a link table associated with the image frame to record the kernel sequence.

3. The method of claim 2, wherein the generating the link table associated with the image frame includes generating a horizontal link table and a vertical link table.

4. The method of claim 1, further including generating a kernel switch lookup table based on the defined kernel patterns, wherein the kernel switch lookup table records kernel switches.

5. The method of claim 4, further comprising updating the kernel switch look up table during image processing and updating a kernel pipeline to prevent data loss at a kernel boundary.

6. The method of claim 1, further comprising defining a plurality of kernel descriptors associated with the image frame, including:

partitioning the image frame into groups of horizontal pixel lines and vertical pixel lines to define a plurality of image frame portions, each having a respective one of the kernel descriptors;

and

defining a plurality of vertical descriptors and a plurality of horizontal descriptors, wherein each kernel descriptors includes a respective vertical descriptor and a respective horizontal descriptor, and defines kernel attributes associated with one of the plurality of image frame portions.

7. The method of claim 6, wherein the defining the plurality of horizontal descriptors includes, for each descriptor:

associating a kernel with the image frame portion;

defining a kernel width associated with the image frame portion; and

determining a loop characteristic associated with the image frame portion in a horizontal direction.

8. The method of claim 6, wherein defining the plurality of vertical descriptors includes, for each vertical descriptor:

associating a link table with the image frame portion;

defining a kernel height associated with the image frame portion; and

determining a loop characteristic associated with the image frame portion in a vertical direction.

9. The method of claim 1, wherein the processing the image frame using the defined kernel patterns includes:

fetching a vertical link table associated with the image frame;

fetching a vertical descriptor associated with the vertical link table;

fetching a horizontal link table associated with the vertical descriptor;

fetching a horizontal descriptor associated with the horizontal link table;

fetching a kernel associated with the horizontal descriptor; and

determining a kernel switch boundary indicating where an image frame portion associated with the kernel ends.

10. The method of claim 9 wherein the processing further includes processing, using the kernel, an image frame portion associated with the vertical descriptor and the horizontal descriptor.

11. The method of claim 1, wherein a kernel includes a plurality of operations to be performed on an image pixel.

12. A non-transitory computer readable medium encoded with instructions that when executed by a processor cause execution of a method for processing an image, wherein the processor can dynamically switch among multiple kernels, the method comprising:

defining a kernel pattern associated with an image frame frame by

grouping image pixels into a plurality of kernel blocks, wherein each kernel block has a respective kernel for performing at least one operation on image pixels in the kernel block, and

determining a kernel sequence for the respective kernels; and

processing the image frame using the defined kernel pattern.

13. The non-transitory computer readable medium of claim 12, wherein defining the kernel pattern associated with the image frame includes generating a link table associated with the image frame to record the kernel sequence.

14. The non-transitory computer readable medium of claim 12, wherein the method further comprises generating a kernel switch lookup table based on the defined kernel patterns, wherein the kernel switch lookup table records kernel switches.

15. The non-transitory computer readable medium of claim 12, wherein the processing includes:

fetching a vertical link table associated with the image frame;

fetching a vertical descriptor associated with the vertical link table;

fetching a horizontal link table associated with vertical descriptor;

fetching a horizontal descriptor associated with the horizontal link table;

fetching a kernel associated with the horizontal descriptor;

determining a kernel switch boundary indicating where an image frame portion associated with the kernel ends; and

processing the image frame using the kernel.

16. The non-transitory computer readable medium of claim 12, further comprising updating the kernel switch look up table during image processing and updating a kernel pipeline to prevent data loss at a kernel boundary.

17. An apparatus for processing an image by dynamically switching among multiple kernels, comprising:

a stream controller that includes;

a kernel storage memory for storing a plurality of defined kernel patterns associated with an image frame, wherein a kernel pattern includes kernel blocks, each including a group of image pixels and having a kernel for performing an operation on image pixels in the group, and a kernel sequence for the respective kernels, and

a kernel look up table (LUT) memory for storing a kernel switch lookup table associated with the plurality of defined kernel patterns, wherein the kernel lookup table records kernel switches;

wherein the stream controller is operable to direct execution of kernels on the image frame according to the plurality of defined kernel patterns and the kernel switch lookup table.

18. The apparatus of claim 17, wherein the stream controller further includes at least two kernel buffers for storing the kernels to be performed on the image frame.

19. The apparatus of claim 17, further including an image processor that includes a stream processing unit, wherein the stream controller is operable to direct the stream processing unit to execute the kernels on the image frame according to the defined kernel patterns.

20. The apparatus of claim 17, further including a compiler operable to generate the kernel switch look up table based on the plurality of defined kernel patterns.